The Future of Software Quality Assurance

This is why iSQI will continue to be a sought-after partner for further training and certification in the years to come. In the next 15 years, the quality of the software tester profession improved tremendously.

About the Editor

Contributors

She is the president of the Association for Software Quality and Education (ASQF), a member of the German National Academy of Science and Technology (acatech) and the Münchner Kreis e.V. He is also a member of the advisory board of ASQF ("Arbeitskreis Software-Qualität und -Fortbildung e.V.").

1 A Vicious Circle

Furthermore, change-driven testing uses test-gap analysis to automatically identify test gaps, that is, code changes that do not have tests. We call the idea of ​​aligning our testing with the changes in application code Change-Driven Testing.

2 Test Intelligence

In practice, we see that about half of the changes can escape even strict testing processes [1,2]. And as testers strive to make test suites more comprehensive by adding additional tests, they also add to the suites' runtime, jeopardizing the benefits of automated tests and CI.

Test Intelligence

Version-Control Systems

New methods are red, modified methods are yellow, and methods that remain unchanged are gray.

Ticket Systems

The size of the box is proportional to the lines of code in that method. Methods that were added by a change are highlighted in red, and methods that were changed are highlighted in yellow.

Profilers

3 Change-Driven Testing

Test-Impact Analysis

It combines relevant code changes (see Section 2.1) with test duration and coverage by test (see Section 2.3).

Test-Impact Analysis

Test-Gap Analysis

Test-Gap Analysis

Limitations

Although the effects are usually small, this means that TIA cannot guarantee that the affected tests will always find all the errors that the entire test suite can find. We therefore recommend that you run the full suite of tests regularly to ensure that nothing is missed and that test run time and coverage information is up to date.

4 Using Change-Driven Testing

Testing with TIA

In our experience, TGA usually reveals substantial test gaps and therefore enables significant improvements to the testing process. To get this data, we extended the CI environment with profilers that record test-by-test coverage in both the front-end JavaScript code and the back-end Java code, and ran the entire test suite once, which took about 45 minutes.

Testing with TGA

Based on this data, TIA now determines a list of six tests affected by the changes: five regression tests that cover parts of the changed code, and a new test that came with the changes. However, since the TGA is aware of the chronological order of changes and test coverage, it will re-report the test gap if the functionality changes at some point in the future.

Fig. 7 The gain from using test-impact analysis. Two consecutive CI runs using TIA plus correcting a mistake take less time than one CI run with Teamscale ’s full test suite

Closing the Loop

As our developer opted for manual exploratory testing, there is no regression testing for this particular functionality. Thanks to this safety net, it is reasonable to choose a quick manual check instead of writing an automated UI test or a manual test for code that is unlikely to ever change again.

5 Adapting Change-Driven Testing

6 Conclusion

1 Introduction

This chapter first looks at some of the key factors affecting businesses today and how the IT industry has responded. Based on those factors, the chapter discusses some of the key testing competencies required of an NG tester.

2 Forces That Drive Businesses

These roles are now well established which is a credit to those organizations and a great improvement on the "dark" days of the past where being a tester was not always considered a particularly attractive and challenging career to pursue. An NG tester can be thought of as someone with a specific set of testing competencies that fit an organization or project.

3 Trends in IT 3.1 Solutions in IT

Practices in IT

4 What Competencies Will Be Needed in Testing?

These competencies, such as the ability to communicate effectively with a wide range of stakeholders, are represented by the horizontal part of the T. Specialized competencies, such as the ability to set up a test environment for testing mobile applications, are represented by the vertical part of the T.

5 General Competencies

• Testing Essentials
• Risk-Based Testing
• System Architectures
• Communication
• Automation
• Adapting to Different Lifecycle Approaches

As an established and well-understood approach, the use of risk-based testing is an essential general competency. Test automation skills are no longer considered a specialty; are an essential skill, especially when agile and DevOps approaches are used in a project.

6 Specialist Competencies

Testing Big Data (BD) Applications

Data analytics performance to deliver "instant" results to the business - Scalability of applications to enable potentially large volumes of data. Data currency, backup and recovery capabilities must be tested to ensure high data quality.

Testing Applications Using Connected Devices

Can the system withstand a wide range of different exception conditions, such as loss of connectivity, intermittent signals, and hardware or software failures in specific system components. Design and perform tests that focus on specific non-functional quality characteristics of applications and systems that use connected devices.

Testing Artificial Intelligence (AI) Applications

Detecting situations where too many matches are found and potentially incorrect answers are given to requests (over-fitting). Because AI applications constantly "learn" and update their knowledge bases, determining expected results is difficult.

7 The Skills Framework

• Competency Descriptions
• Competency Levels
• Assessing Competencies
• Profiles

Full level: At this level, a person can fully demonstrate the competence in a project without support. The criteria to be achieved are similar to those described for the full level, except that the length of the experience is longer, the portfolio more extensive and the pre-determined practical task more complex.

8 Conclusion

The images or other third-party materials in this chapter are included in the chapter's Creative Commons license, unless otherwise indicated in a credit line for the material. With DevOps, changes are needed to make sure that testing really focuses on the quality of the service delivered.

2 What Is DevOps?

• Customer-Centric Action
• Create with the End in Mind
• End-to-End Responsibility
• Cross-Functional Autonomous Teams
• Continuous Improvement
• Automate Everything You Can

Teams are responsible for the entire SDLC and they must be aware of and respond to any changes the service undergoes. Team sharing is important because it allows engineers to grow in the horizontal bar of the T-shaped model thanks to the knowledge and skills of their team members.

3 Testing in DevOps

Scope

Business processes can cover multiple services, and end-to-end testing will test the services of multiple teams. With this approach, it is still possible to perform end-to-end testing with DevOps teams.

Customer-Centric Testing

In certain situations, it can and will be more successful if one team becomes responsible for end-to-end testing when covering multiple services. The end-to-end testing team should be responsible for making sure the tests are executed, but the DevOps teams are still responsible for their services in those tests.

Specialized Testing

It may be possible for an engineer to become powerful in both security and performance testing. On the other hand, it must be clear to DevOps teams what is expected of the tests.

4 Automation

Test Automation

Business-oriented tests that support the team, like functional tests, can be automated or performed manually. However, technology that faces test criticism of the project mostly requires tools and is therefore already automated.

Continuous Testing

These functional tests in DevOps should be automated, based on the DevOps principle “Automate everything you can” [3]. In DevOps teams, it should enable all team members to deploy any version in any environment with proper controls.

Monitoring

Continuous testing can act as a continuous process where every time the pipeline is started, tests are executed. Tests can act as go or no-go points in the pipeline to move to the next step in the process.

5 The Role of a Test Engineer

T-Shaped, Test Shaped

Soft Skills

6 Conclusions

Kim, G., Behr, K., Spafford, G.: The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win. International Software Testing Qualifications Board: Certified Tester Foundation Level Syllabus Version 2018 Version.https://www.istqb.org/downloads/send/51-ctfl2018/208-ctfl-2018-syllabus.html(2018).

Teaching Testers to Think for Themselves

1 Introduction 1.1 Background

Stakeholders

Program members are the group of professional software testers representing software companies in the south of Ireland.

Initiation

2 Why a New Tester Skills Program?

• Existential Crisis for Testers
• The Drive to Digital
• Waterfall Thinking Won’t Work with Continuous Methods
• Separating Thinking from Logistics
• Logistics: We Don’t Care
• Without Logistics: The New Model for Testing
• Rethinking Status
• The (Current) Problem with Certification

We don't care who you are – – developer, tester, business analyst user – – the principles of testing are universal. Curricula do not teach thinking skills. The word think appears once in 142 pages of the Foundation and Advanced Test Analyst syllabi.

Fig. 1 The new model for testing

3 The Tester Skills Program

• Skills Focus
• New Tester Behaviours
• TSP Is a Learning and Development Scheme
• The Skills Inventory
• Program Member Challenges
• Structure of the Foundation Scheme

If you look at the syllabuses of the certification schemes (e.g. Foundation and Advances Test Analyst), the table of contents mainly contains what we have called logistics. The certification schemes learn a lot from the things we say we don't care about. The program members' initial work was to define the range of skills required of a test practitioner.

4 The Future

• Engaging Competent Training Providers to Deliver TSP
• Could the TSP Be the Basis of Certification?
• Classroom, Online Instructor-Led or Online Teaching?
• Proliferating the TSP

If material is not included in the chapter's Creative Commons license and your intended use is not permitted by law or exceeds the permitted use, you must obtain permission directly from the copyright holder. Summary The development of autonomous vehicles is currently being promoted massively, not least in the German car industry, under very high investments.

1 Motivation

This chapter therefore addresses the question of how the testing of future autonomous systems will differ from the testing of software-based systems of the current character and makes some suggestions for the associated further development of the test procedure. This chapter therefore addresses the question of how the testing of future autonomous systems will differ from the testing of software-based systems of the current character and makes some suggestions for the associated further development of the test procedure.

2 Autonomous Systems

Autonomy and Autonomy Levels

The extent to which an autonomous system fulfills these properties (sensing, perceiving, analysing, etc.) can vary greatly. Automated vehicle guidance without the expectation that the driver will respond to a request to intervene.

Capabilities of Fully Autonomous Systems

The framework describes how a metric-based classification can be performed within each of these dimensions and how an overall system rating ("Contextual Autonomous Capability") can be determined from this.

3 Safety of Autonomous Systems

Safety in Normal Operation

The situation in which a plastic or wooden object is located too close to or above the cooking plate may be considered more unsafe, as the risk of toxic fumes or fire from burning plastic or wood is potentially present. The risk is not present in a situation in which these objects are located far from the cooking plate (with a certain safety margin), regardless of the state of the cooking plate.

Safety in Failure Mode

A service robot is instructed to "water the plants". In this regard, it is assumed that a plug has fallen into a plant pot. If a self-driving car were to stop "in the middle of the road" in the event of a critical sensor failure, the car would increase the risk it poses rather than reduce it.

4 Testing Autonomous Systems

• Quality Characteristics and Test Topics
• Implications of Learning
• New Test Method: Scenario-Based Testing
• Requirements for the Test Process

Performance: Time behavior of the system and its components and resource consumption should be checked. Compatibility: This concerns interoperability between the components of the system itself (sensors, controllers, actuators) as well as compatibility with external systems.

5 Conclusion and Outlook

High Level Independent Expert Group on Artificial Intelligence - Established by the European Commission, Ethical Guidelines for Trusted Artificial Intelligence, 4/2019.https://ec.europa.eu/futurium/en/. IEC Functional Safety of Electrical/Electronic/Programmable Electronic Safety Systems - Parts 1 to 7, https://www.iec.ch/functionalsafety/.

Further Reading

When you read about the extremely rapid developments currently taking place in information technology (eg, computers) and operational technology (eg, factories), with artificial intelligence, machine learning, robots, chatbots, self-driving cars, autonomous vacuum cleaners and many other intelligent machines.

1 Testing Of and With Intelligent Machines

Then we will focus on testing in intelligent machines, meaning the use of intelligent machines to support testing.

2 Testing Of Intelligent Machines

Six Angles of Quality for Machine Intelligence

With the rise of machine intelligence and robotics, test methods are being used again to test basic information processing components of intelligent machines. And now we are seeing the rapid rise of machine intelligence, bringing new challenges for test engineers.

Fig. 1 The six angles of quality in the digital age

Quality Is Defined and Tested Using Quality Characteristics

• Intelligent Behavior
• Morality
• Personality
• Usability

A robot must be aware of the people around and know when a person wants to interact with the robot. In chatbots it is important that the conversation is natural, but also specific to the purpose of the chatbot.

Fig. 2 The ISO25010 standard with quality characteristics in gray and our extension with quality characteristics for intelligent machines in red

3 Testing With Intelligent Machines

• Models Help Quality Forecasting
• Robots Testing Physical Machines
• Test Execution in a Virtual Engineering Environment
• Beware of Testing AI with AI

By manipulating test environments based on real-time data trends, various situations can be calculated. Just like with GUI test automation, standard building blocks can be used in the physical domain as well.

Fig. 3 Testing is moving from a reactive activity (test execution) through quality monitoring for operational systems towards quality forecasting where faults are predicted so that they can be fixed even before anyone notices a failure

Testing

1 Acceptance Testing, Part 1: Introduction

Imagine, you find a serious bug in the acceptance testing phase - what are you supposed to do. This affects the development of the software, the software test cases created and finally the conception of the project.

Table 1 shows the status of the IT projects in the last 5 years—CHAOS Report 2015.

2 Acceptance Testing, Part 2: The Process

• Types of Acceptance Testing
• Alpha Testing
• Beta Testing
• Main Differences Between Alpha and Beta Testing

Our advice is to be very careful when making a beta release and make sure you've done some alpha testing beforehand. To be more precise in distinguishing alpha and beta tests since they are the most common, here is a comparative table (Table 4).

3 Acceptance Testing, Part 3: Approaches

• Primary Approach to Acceptance Testing
• Basic Steps to Organize a Well-Done Acceptance Phase
• Approaches to Create Acceptance Test Cases
• Roles and Responsibilities

Tested in the "real world" with "real customers" and feedback can cover every element of the product. The downside of business process/workflow test cases is that data related testing is not in its scope.

Table 4 Main differences between alpha and beta testing

4 Acceptance Testing Part 4: Conclusions

• The Execution: What to Avoid When the Acceptance Testing Is Performed?
• Involving the End Users
• Acceptance Testing Performed by the Internal QA Team
• What Are the Main Reasons the Acceptance Testing Phase Often Fails?
• Best Option for Successful Acceptance Testing
• To Outsource Acceptance Testing or Not?

So we got down to the nitty-gritty of finding out what are the main reasons why the acceptance testing phase often fails. That way, after planning everything carefully and considering outsourcing options, you'll have a really good release at the end of the acceptance testing phase.

Fig. 2 Outsourcing acceptance testing: onsite and offshore tasks

The Challenge of Working with Distributed Testing Teams and Performing a High-Quality Work

2 Selection of an Offshore Team

3 About Training

Determine how the lectures will be conducted, who will lead them and where the teaching materials will be available. Therefore, to achieve better results, it is crucial to consider the "face-to-face" method for at least part of the training.

4 Define How Communication Will Be

Let's note that the main goal of the training is to give the teams the general context of each project and align them to a common understanding and way of working. On the other hand, it is fundamental to define a central point of the offshore team, that is the point of contact with whom you can review the progress of tasks, problems and delays that may occur.

5 How to Maximize Productivity of the Offshore Team?

• About Requirements
• How to Design Test Cases
• About Tests Execution
• About Tools

Setting up frequent short calls can perfectly prevent long email chains where it's easy to lose track and generate confusing information without resolving the issues. With regard to test cases, it is important to define how defects are reported and what information the defect report will contain.

6 Retrospective

All the information generated must be registered and available to all project stakeholders, either for viewing only or for viewing and updating.

7 Some Conclusions

Imagine a situation: it's the end of the sprint, most of the stories are done from a development perspective, and they're in the QA column on the board. Still, with the Sprint demo, they still can't report complete success: developers completed a few more stories on the last day of the Sprint and were unable to complete testing.

2 The Shift in Testing Paradigms

Second, we will expand our scope of thinking about quality as something related to the technological side of the product. It is therefore in the collective ownership and responsibility of the whole team to embrace the different aspects of quality and engage in activities that contribute to the different types of testing.

3 Investment in Automated Testing

In the agile paradigm, we need to invert the pyramid as shown on the right side of the figure. The top layer tests end-to-end scenarios through the system's user interface.

Fig. 2 Software product lifecycle

4 Transitioning to Agile Testing

However, please note that we can also optimize manual testing strategies to get the most value out of the effort spent there. A release later, we began expanding our strategy to include automated unit and integration testing to cover those obsolete parts that we needed to touch and rework as part of the new development effort.

5 The People Perspective

The example I showed, in combination with the concepts discussed in the previous sections, can give you some ideas on how to start applying some of the concepts of Agile testing. As in the case of ethnic cleansing in Myanmar, where tens of thousands of Rohingya were killed and a million fled.

2 An Introduction to AI for Testers

• AI Is Black Box Development
• Machine Learning and Neural Networks
• Algorithm = Data + Code + Labels
• Fuzzy Logics and Mathematics
• Development and Correction
• Overall Version Evaluation and Metrics

In this way, the neural network creates models of the labels: these reflect differences between a cat and a father that the neural network has identified based on the training data. The algorithm is a product of the neural network, but based heavily on the training data (the examples) and the targets (the labels).

3 Risks in AI

• Bias
• Selection Bias
• Confirmation Bias
• Under-fitting
• Over-fitting
• Outliers
• Confounding Variables
• Over-confidence in AI
• Under-confidence in AI
• Traceability

For certain AI issues, such as subordination, expanding the training data will most likely improve the system. We want to know the overall performance of the system compared to other versions.

4 Testing AI

• Review of the Neural Network, Training Data and Labelling
• Identifying Users
• Profiling Users
• Creating Test Cases
• Test Data
• Metrics
• Weighing and Contracts
• Test Automation
• Overall Evaluation and Input for Optimising
• Example of AI Test Framework (Fig. 2)

Possible negative side effects of existing contracts should also be part of the test. But even if the AI ​​system is not continuously trained and the versions of the system are stable, the changing context requires continuous training.

5 Conclusions

Additionally, traceability of goals, functions, and responsibilities must become part of any software engineering and documentation. The following is an overview of the current understanding of ethical principles in software engineering and draws conclusions in the direction of responsible software engineering.

2 Software and Recent Software Quality Requirements

Rather, there are surveys for specific areas of software applications or software development tools, such as in software-defined networks [7], for user interfaces [8] or for software documentation [9]. However, as software technologies evolve, the understanding of software quality must evolve as well.

3 Software Criticality and the Need for Responsible Software Engineering

Finally, such responsible software engineering (see Figure 2) could be promoted by the Weizenbaumian Oath [28] to reflect professional ethics for the sustainable design, development, operation and maintenance and use of software and software-based systems. With the Weizenbaumian Oath, all technology communities could commit to general principles that guide the development and use of software and software-based systems.

Fig. 1 Elements of software-based systems [23]. Sensors are part of the Internet of Things and generate different kinds of data such as measurements, series of measurements or data streams.

4 Ethical Principles in Responsible Software Engineering

Document the goals and expected impact: The goals and expected impact of using an algorithmic system should be documented and evaluated before implementation. Security guarantee: The security of an algorithmic system must be tested before and during its implementation.

Fig. 2 Constituents of responsible software engineering

5 Outlook

Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems (1992) 9. Paper presented at the Proceedings of the 2nd international conference on Software engineering, San Francisco, California, USA (1976).

2 Automated Testing

At its core, mutation testing is the practice of running a test or set of tests on many versions of the software under test—a so-called mutant. The purpose of testing is usually to determine which defects are detected by the test procedures, which cause defects.

Fig. 1 In this example, Mutant 1 doesn’t cause any tests to pass, indicating that the injected faults is not detected, hence there is a fault in the test coverage

3 Code Coverage and Mutation Operators

Measuring the code coverage of your automated tests is great, but it's only part of the picture. Running the software under test via the mutation operator can cause many changes to the code.

Table 1 Mothra operators Type Description

4 Mutation Testing Challenges and Strategies

In order to apply horizontal scalability, it is necessary to consider this when choosing the appropriate mutation testing tools with support for such approaches. It can also occur when software is less deterministic and when it is difficult to understand whether the failure to detect a mutation is actually meaningful.

5 Mutation Testing Tools

Similarly, SonarQube [10], the popular code quality monitoring package, has plugins that allow you to display detailed results from your mutation test. Some mutation testing engines also offer IDE plugins that speed up the feedback loop and let you see the results in the context of your code.

6 Other Applications of Mutation Testing

How to incorporate the mutation testing into existing infrastructure: various build systems, test frameworks, CI pipelines, IDEs, etc.—Alex Denisov. Fuzz testing, which mutates the input domain of an application, can also be considered a type of mutation testing.

7 Conclusion

Although mutation testing has been around for a while, it's solidly building support in the engineering community, and it's clearly providing useful information that just isn't available anywhere else. Saxena, R., Singhal, A.: A Critical Review of Mutation Testing Techniques and Obstacles, in 2017 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, pp.

2 Embracing Quality

The introduction of quality function (QFD) pioneered by Shigeru Mizuno and Yoji Akao sensitized designers to customer needs and this practice helped introduce the term "voice of the customer". The QFD approach brings customer satisfaction early in the production or development processes [1]. 2 Design thinking as a means of achieving user-informed development and quality [in the style of visual thinking].

Fig. 1 Relationship of embracing quality to development activities

3 Introducing Design Thinking

Fundamental Concepts

Design thinking is a promising method for providing products and services based on a development process based on user input. Design thinking relies heavily on a team's ability to work, explore, fail and learn together.

Design Thinking Resources

As the team makes deeper commitments to design alternatives, the degree of fidelity increases, which is reflected in the level of engineering investment and operational sophistication of the prototype. The team may seek to test a final design under conditions close to real-world conditions prior to design submission and design closure.

Design Thinking Approaches

The design thinking team needs the support of the sponsoring organization in terms of budget, approach, schedule and outcome expectations. One popular layout of the approach consists of the stages Empathize, Define, Ideate, Prototype and Test [11].

Fig. 3 Explanation of the diamond metaphor

4 Design Thinking’s Role in Quality

The design result provides an established set of user-oriented requirements to be implemented and tested for accepted production quality. By having senior developers participate in the design thinking project, different development conventions and tool chains used in standard development projects are more likely to be used during prototype implementation, making reuse more feasible.

Fig. 5 Design thinking is central to the design candidate development processes

5 Challenges of Design Thinking

Immersive quality is a convergence quality that results from the integration of the user with a product or service. The greatest impact the design team can have on the user is through the design of user experiences and instructional resources.

Capabilities in Hispanic America

Challenges and Prospects

Engineering is their main subject, and they focus even less on software quality and/or software testing. The chapter concludes with a look at future challenges and promising prospects for the future of software quality and software testing in Hispanic America.

2 Background

Today, due to the spread of the Internet (and the World Wide Web) and mobile communication, digital technologies have become pervasive in Latin America. 442 million unique mobile subscribers across Latin America and the Caribbean, accounting for 68% of the [subcontinent's] population” [10].

3 Skill Sets

Software Engineering

By 2000, Finkelstein and Kramer defined software engineering as “the branch of systems engineering concerned with the development of large and complex software-intensive systems. IEEE has been developing the most comprehensive family of civil software engineering standards since the 1980s.

Human Competencies

In terms of personal competencies, two professional project management associations, the International Project Management Association (IPMA) and the Project Management Institute (PMI), have developed competency models that include 'behavioral', 'people' or 'personal' competencies: the IPMA Individual Competency Baseline and the PMI Project Manager Competency Development Framework.

4 Industry

To overcome the limitations of new hires in software testing and quality assurance, Softtek has an extensive training and career development program that also includes bridges to industry-recognized professional and technical certifications. Notable among Latin American companies is Choucair Testing http://www.cho ucairtesting.com, founded in 1999 by María Clara Choucair, the company focuses on software testing and quality assurance services.

5 Impact of Professional Certification Schemes