We may do this for both the functional features of the software (for example, to print a report correctly) and for the non-functional software requirements and features (for example, to print a report quickly enough). Testing can give confidence in the quality of the software if it finds few or no defects, provided we are satisfied that the testing is sufficiently rigorous.
How much testing is enough?
Deciding how much testing is enough should consider the level of risk, including technical and business risks related to the product and project constraints such as time and budget. Due to the limitations in the budget, the time and in testing we have to decide how we will focus our testing based on the risks.
WHAT IS TESTING?
Risk assessment and management is one of the most important activities in any project, a key activity and the reason for testing. In addition, testing should provide stakeholders with sufficient information to make informed decisions about releasing the software or system under test to the next development step or handover to customers.
1 Recall the common objectives of testing. (Kl)
Clients and project managers will want to spend an amount on testing that provides them with a return on investment. We do this by aligning the testing we do with the risks to the customers, the stakeholders, the project and the software.
2 Describe the purpose of testing in software development, maintenance
Full testing – even when required by clients and project managers – is simply not something they can afford. The effort invested in quality assurance and testing activities must be commensurate with the risks and costs associated with the project.
Testing our one-digit field with values 2, 3, and 4 makes our tests more thorough, but it doesn't give us more information than if we just tested with the value 3. Instead, we need a testing approach that provides the right amount of testing for this project, these customers (and other stakeholders), and this software.
- The driving test - an analogy for software testing
- Defining software testing
- Software test and driving test compared
- When can we meet our test objectives?
- Focusing on defects can help us plan our tests
- The defect clusters change over time
- Debugging removes defects
- Is the software defect free?
- If we don't find defects does that mean the users will accept the software?
- TESTING PRINCIPLES
The time for the test is limited, so it is not a complete test of the driver's skills, but it is representative and allows the examiner to make a risk-based decision about the driver. The definition begins with a description of testing as a process and then lists some objectives of the testing process.
1 Explain the fundamental principles in testing. (K2)
FUNDAMENTAL TEST PROCESS
1 Recall the fundamental test activities from planning to test closure activities and the main tasks of each test
- Introduction
- Test planning and control
- Test analysis and design
- Test implementation and execution
- Evaluating exit criteria and reporting
- Test closure activities
- THE PSYCHOLOGY OF TESTING
- Independent testing - who is a tester?
- Why do we sometimes not get on with the rest of the team?
We will have to take steps where necessary to achieve the goals of the project. For the driving examiner this could mean changing the test condition 'junctions' to 'taking the route down Mayfield Road to the junction with Summer Road and asking the driver to turn left into Summer Road and then right into Green Road, with the expectation that the driver checks mirrors, signals and maneuvers correctly, while remaining aware of other road users.' We may need to automate some tests using test harnesses and automated test scripts.
CHAPTER REVIEW
SAMPLE EXAM QUESTIONS
EXERCISE: TEST PSYCHOLOGY
EXERCISE SOLUTION
SOFTWARE DEVELOPMENT MODELS
1 Understand the relationship between development, test activities and work products in the development life cycle and give examples based on
2 Recognize the fact that software development models must be adapted to the context of project and product characteristics. (Kl)
3 Recall reasons for different levels of testing and characteristics of good testing in any life cycle model. (Kl)
V-model
Defects were discovered too late in the life cycle, as testing was only included at the end of the project. Test levels can be combined or reorganized depending on the nature of the project or system architecture.
Iterative life cycles
Subsequent increments will need to be tested for the new functionality, regression testing of the existing functionality, and integration testing of both new and existing parts. Early market presence will also mean that validation testing is performed at each increment, thereby providing early feedback on the business value and suitability for use of the product.
Testing within a life cycle model
- Component testing
- Integration testing
- System testing
- Acceptance testing
- TEST TYPES: THE TARGETS OF TESTING
System functional requirements testing begins by applying the most appropriate specification-based (black box) techniques for the aspect of the system to be tested. The objective of acceptance testing is to establish confidence in a system, part of a system or specific non-functional characteristics, e.g.
1 Compare four software test types (functional, non-functional, structural
2 Recognize that functional and structural tests occur at any test level
3 Identify and describe non-functional test types based on non- functional
4 Identify and describe test types based on the analysis of a software
5 Describe the purpose of confirmation testing and regression testing
- Testing of function (functional testing)
- Testing of software product characteristics (non-functional testing)
- Testing of software structure/architecture (structural testing)
- Testing related to changes (confirmation and regression testing)
A second objective for testing is to test the quality characteristics, or non-functional attributes of the system (or component or integration group). All the test cases in a regression test suite will be executed every time a new version of the software is produced, and this makes them ideal candidates for automation.
Maintenance of a regression test suite should be performed so that it evolves over time in line with the software. Sometimes a regression test suite of automated tests can become so large that it is not always possible to run them all.
1 Compare maintenance testing (testing an operational system) to testing
This choice must be made in light of the latest changes made to the software. It may be possible and desirable to eliminate some test cases from a large regression test suite, for example if they are repetitive (tests that exercise the same conditions) or can be combined (if they are always run together).
As new functionality is added to a system, new regression tests should be added, and as old functionality is changed or removed, regression tests should also be changed or removed. Another approach is to eliminate test cases that have not found a defect for a long time (although this approach should be used with some caution!).
If all the tests must be performed manually, it may not be possible to perform all of them each time the regression package is used.
2 Identify reasons for maintenance testing (modifications, migration and
3 Describe the role of regression testing and impact analysis in mainte
Impact analysis and regression testing Usually maintenance testing will consist of two parts
In addition to testing what has been changed, maintenance testing involves extensive regression testing on parts of the system that have not been changed. During impact analysis, together with stakeholders, a decision is made as to which parts of the system may be inadvertently affected and therefore need careful regression testing.
Triggers for maintenance testing
From Section 2.4, you should be able to compare maintenance testing with new application testing. Finally, you should be able to describe the role of regression testing and impact analysis in maintenance testing.
CHAPTER THREE
Static techniques
REVIEWS AND THE TEST PROCESS
1 Recognize software work products that can be examined by different static techniques. (Kl)
2 Describe the importance and value of considering static techniques for the assessment of software work products. (K2)
3 Explain the difference between static and dynamic techniques. (K2)
Because static testing can begin early in the life cycle, early feedback on quality issues can be obtained. In conclusion, static testing is a very suitable method to improve the quality of software work products.
1 Recall the phases, roles and responsibilities of a typical formal review
2 Explain the differences between different types of review
- Phases of a formal review
- Roles and responsibilities
- Types of review
- Success factors for reviews
The specific goals of a walkthrough depend on its role in the creation of the document. During technical reviews, defects are found by experts, who focus on the content of the document.
3 .3 STATIC ANALYSIS BY TOOLS
Follow all formal rules until you know why and how to change them, but only make the process as formal as the project culture or maturity level allows. Each step of the process is clear, but it takes experience to do them correctly.
1 Describe the objective of static analysis and compare it to dynamic
Continuous improvement of processes and supporting tools (eg checklists) based on the ideas of the participants ensures the motivation of the engineers involved. The costs must of course be monitored, but the benefits, especially if no problems arise in the future, should be visible by quantifying the benefits and costs.
2 Recall typical defects identified by static analysis and compare them to
3 List typical benefits of static analysts. (Kl)
4 List typical code and design defects that may be identified by static
Coding standards
It can even be put the other way around: buy a static code analyzer and declare (a selection of) the lines in it as your coding standard. Without such tools, enforcement of an encryption standard in an organization is likely to fail.
Code metrics
Typically, an encoding standard consists of a set of programming rules (e.g., "Always check the bounds on an array when copying to this array"), naming conventions (e.g. There are three main reasons for this: the number of rules in an encoding standard is usually so large, that no one can remember them all; some context-sensitive rules that require multi-file reviews are very difficult for people to check; and if people spend time checking coding standards in reviews, it will deter them from making other mistakes that they might otherwise make had to find, making the review process less efficient.
- Code structure
- Which of the following artifacts can be examined by using review
- Which statement about the function of a static analysis tool is true?
- Which is not a type of review?
- What statement about reviews is true?
- What is the main difference between a walkthrough and an
- Which of the following characteristics and types of review
- Led by the author 2. Undocumented
- No management participation
- Led by a trained moderator or leader 5. Uses entry and exit criteria
- Which of the following statements about early test design are true
- Defects found during early test design are more expensive to fix
- Early test design can find defects
- Early test design can cause changes to the requirements
- Early test design takes more effort
- Static code analysis typically identifies all but one of the following
You should be able to recognize software work products that can be examined by static testing. You should be able to describe the main features of static analysis and recall typical defects that can be found using static analysis.
CHAPTER FOUR
IDENTIFYING TEST CONDITIONS AND DESIGNING TEST CASES
1 Differentiate between a test design specification, a test case specification and a test procedure specification. (Kl)
4 Translate test cases into a well-structured test procedure specification at a level of detail relevant to the knowledge of the testers. (K3)
- Introduction
- Formality of test documentation
- Test analysis: identifying test conditions
- Test design: specifying test cases
- Test implementation: specifying test procedures or scripts The next step is to group the test cases in a sensible way for executing
The test conditions chosen will depend on the test strategy or detailed test method. Test conditions should be traceable back to their sources in the test foundation - this is called traceability.
1 Recall reasons that both specification-based (black-box) and structure-
2 Explain the characteristics and differences between specification-based
- Introduction
- Static testing techniques
- Specification-based (black-box) testing techniques
- Structure-based (white-box) testing techniques
- Experience-based testing techniques
- Where to apply the different categories of techniques Specification-based techniques are appropriate at all levels of testing (compo-
- SPECIFICATION-BASED OR BLACK-BOX TECHNIQUES
The first of the dynamic testing techniques we will look at are the specification-based testing techniques. Structure-based testing techniques (which are also dynamic rather than static) use the internal structure of the software to derive test cases.
1 Write test cases from given software models using the following test
For example, covering menu options or major business transactions may be the structural element in system or acceptance testing. Experience-based techniques are used to complement specification- and structure-based techniques, and are also used when there is no specification, or if the specification is inadequate or outdated.
This may be the only type of technique used for low-risk systems, but this approach can be especially useful under extreme time pressure; this is in fact one of the factors that lead to exploratory testing.
2 Understand the main purpose of each of the four techniques, what level
3 Understand the concept of use case testing and its benefits
- Equivalence partitioning and boundary value analysis Equivalence partitioning
- Decision table testing
- State transition testing
- Use case testing
Therefore, we have a value on each side of the boundary (but the boundary itself is not a value). A use case is a description of a particular use of the system by an actor (a user of the system).
1 Describe the concept and importance of code coverage. K2) 2 Explain the concepts of declaration and decision coverage.
3 Write test cases from given control flows using the following test design
4 Assess statement and decision coverage for completeness
Using structure-based techniques to measure coverage and design tests
Structure-based test design techniques are a good way of generating additional test cases that are different from existing
Statement coverage and statement testing
However, we will only use numbered lines to illustrate the principles of statement (line) coverage. This time the value of C is 70, so we print 'Big C' and we've exercised all six statements, so now the coverage of the statements = 100%.
Decision coverage and decision testing
The dotted line shows where Test 2_1 left off and clearly shows that we have not yet had a test that takes the False output of the IF statement. This now includes both decision results, True (with Test 2_1) and False (with Test 2_2).
Other structure-based techniques
This technique requires the coverage of all conditions that may affect or determine the decision outcome. A good description of the graph theory behind structural testing can be found in [Jorgensen, 1995] and [Hetzel, 1988] also shows a structural approach.
Error guessing
Fallacy is a technique that should always be used as a supplement to other more formal techniques. The success of error guessing is very dependent on the skill of the tester, as good testers know where the defects are most likely to hide.
Exploratory testing
This is why a guesswork approach, used after more formal techniques have been applied to some extent, can be very effective. A structured approach to the error guessing technique is to list possible defects or errors and design tests that attempt to produce them.
1 List the factors that influence the selection of the appropriate test design technique for a particular kind of
- Why are error guessing and exploratory testing good to do?
- How do experience-based techniques differ from specification-based techniques?
- When choosing which technique to use in a given situation, which factors should be taken
- Given the state diagram in Figure 4.6, which test case is the minimum series of valid
This knowledge will itself be influenced by their experience of testing and of the system being tested. The content and style of the documentation will also influence the choice of techniques (if, for example, decision tables or state charts are used, the associated test techniques must be used).
EXERCISES: TEST DESIGN TECHNIQUES
EXERCISE SOLUTIONS
We have assumed that this applies to all travelers for the family card, but for individual travelers only for the over-60s card. Any additional items added to the cart will not change the status (only the total number of items to purchase).
CHAPTER FIVE
Test management
TEST ORGANIZATION
1 Recognize the importance of independent testing. (Kl)
2 List the benefits and drawbacks of independent testing within an organ ization. (K2)
3 Recognize the different team members to be considered for the creation of a test team. (Kl)
4 Recall the tasks of typical test leaders and testers. (Kl)
- Independent and integrated testing
- Working as a test leader
- Working as a tester
- Defining the skills test staff need
- TEST PLANS, ESTIMATES AND STRATEGIES
Similarly there is a big difference in the roles that people play within the test team. They ensure proper configuration management of produced testware and traceability of tests back to the test base.
1 Recognize the different levels and objectives of test planning
2 Summarize the purpose and content of the test plan, test design specifi
3 Recall typical factors that influence the effort related to testing. (Kl)
4 Differentiate between two conceptually different estimation approaches
5 Differentiate between the subject of test planning for a project, for indi
6 List test preparation and execution tasks that need planning. (Kl)
7 Recognize and justify adequate exit criteria for specific test levels and
The purpose and substance of test plans
For some systems projects, a hardware test plan and a software test plan will address different techniques and tools, as well as different audiences. However, since there may be overlap between these test plans, a master test plan that addresses common elements can reduce the amount of redundant documentation.
5.2.2 What to do with your brain while planning tests
- Estimating what testing will involve and what it will cost The testing work to be done can often be seen as a subproject within the larger
- Estimation techniques
- Factors affecting test effort
- Test approaches or strategies
- TEST PROGRESS MONITORING AND CONTROL
This is even more important when applying exit criteria at the end of the project. Finally, increasing the size of the product leads to increases in the size of the project and the project team.
1 Recall common metrics used for monitoring test preparation and execu
If you can use an older system as a model for a new system, you can use a model-based strategy. Furthermore, select testing strategies with an eye to the factors mentioned earlier, the project's schedule, budget and functional constraints, and the realities of the organization and its policies.
2 Understand and interpret test metrics for test reporting and test control
3 Summarize the purpose and content of the test summary report docu
Monitoring the progress of test activities
The status of the test case is shown in column C ("Warn" indicates a test that resulted in a minor error). In addition to test case status, it is also common to monitor test progress during the test execution period by looking at the number of defects found and resolved.
Reporting test status
As a complementary monitoring technique, you can assess the subjective level of confidence that test takers have in the test items. In addition to including the type of graphs and charts shown earlier, you can discuss important events (especially problematic ones) that occurred during testing, the test objectives and whether they were achieved, the test strategy followed and how well it worked. , and the overall effectiveness of the test effort.
Test control
Test control can include re-prioritizing tests so that we start testing against what is available now. In this case, test control may include requiring that the programmers making the fixes fully retest the fixes before checking them into the code repository for inclusion in a test build.
For cost reasons, performance testing is normally performed in the production environment on weekday evenings after business hours. Although these examples show test control actions that impact testing, the project team may also need to take other actions that impact others on the project.
1 Summarize how configuration management supports testing. (K2)
Due to unexpected high demand for your products, the company has temporarily adopted an evening shift that keeps the production environment in use 18 hours a day, five days a week. For example, suppose that the test completion date is at risk due to a large number of bug fixes that do not pass confirmation testing in the test environment.
Configuration management is a topic that often perplexes new practitioners, but, if you ever have the bad luck to work as a tester
As the project continues, the configuration process and mechanisms must be implemented, and key interfaces with the rest of the development process must be documented. Come test execution time, this will allow you and the rest of the project team to avoid nasty surprises like testing the wrong software, getting uninstallable builds, and reporting unreproducible bugs to code versions that they don't exist anywhere except in the test environment.
Although our description was brief, configuration management is a topic as complex as test environment management. During the project planning phase - and perhaps as part of your own test plan - ensure that configuration management procedures and tools are chosen.
1 Describe a risk as a possible problem that would threaten the achieve
Last but not least, configuration management allows us to map what is being tested into the underlying files and components that make it up. Ideally, when testers receive an organized, version-controlled test release from a change-managed source code repository, it is accompanied by a test item release report or release notes.
For the type of test reports discussed earlier to have any meaning, we need to be able to trace the test results back to what exactly we tested. The release notes are not always so formal and do not always contain all the information indicated.
For example, when we report bugs, we need to report them against something, something under version control. If it's not clear what we found the bug in, programmers will have a very hard time finding the bug to fix it.
5 Describe, using examples, how risk analysis and risk management may
- Risks and levels of risk
- Product risks
- Project risks
- Tying it all together for risk management
However, testing is an activity like the rest of the project and is therefore subject to risks that endanger the project. Mitigation: Take steps in advance to reduce the likelihood (and possibly the impact) of the risk.
A common problem people encounter when organizations first adopt risk-based testing is the tendency to be overly alarmed by certain risks once they are clearly articulated. As with life, the goal of risk-based testing should not be a risk-free project – and in practice it cannot be.
1 Recognize the content of the [IEEE 829] incident report
It is very important to maintain a sense of perspective, a focus on the point of the exercise. What we can achieve with risk-based testing is the marriage of testing with best practices in risk management to achieve a project outcome that balances risks with quality, features, budget, and schedule.
2 Write an incident report covering the observation of a failure during
Make sure you plan to re-evaluate and adjust your risks at regular intervals throughout the project and make appropriate course corrections to the testing or the project itself. Examine the risks by understanding how much of your overall effort can be spent on them.
What are incident reports for and how do I write good ones?
An incident report contains a description of the misconduct observed and classification of that misconduct. Thus, the impact of the problem on the users, customers and other stakeholders is important.
What goes in an incident report?
What happens to incident reports after you file them?
You need to know which parts of the testing process require special attention in test planning. Which of the following test plan elements, while specified during test planning, is evaluated during test execution.
EXERCISE: INCIDENT REPORT
Short description The client environments are not clearly recognized on the screen by the user, and the method of logging into the client environment allows switching between the test and production environments. Impact Rating Priority Low to Medium Severity Very High. does not stop the testing process) (may affect users' business) Detailed description Client environments are not clearly recognized on the screen for the user, and the method of logging into the client environment allows switching between test and production environments.
CHAPTER S I X
TYPES OF TEST TOOL
1 Classify different types of test tools according to the test process activi
2 Recognize tools that may help developers in their testing
- Test tool classification
- Tool support for management of testing and tests
- Tool support for static testing
- Tool support for test specification
- Tool support for test execution and logging Test execution tools
- Tool support for performance and monitoring
- Tool support for specific application areas (Kl)
- Tool support using other tools
Static analysis tools are usually used by developers as part of the development and component testing process. Modeling tools are typically used by developers and can help with the design of the software.
For example, there are web-based performance testing tools as well as performance testing tools for back office systems. There are dynamic analysis tools that focus on security issues, as well as dynamic analysis tools for embedded systems.
1 Summarize the potential benefits and risks of test automation and tool
There are static analysis tools for specific development platforms and programming languages, as each programming language and each platform has distinct features. Tools used by developers during debugging, to help locate defects and check their fixes, are also testing tools.
Commercial toolkits can be bundled for specific application areas, such as web or embedded systems. It's a good idea to look at any type of tool available to you and see how it could be used to support any testing activity.
2 Recognize that test execution tools can have different scripting tech
- Potential benefits of using tools
- Risks of using tools
- Special considerations for some types of tools Test execution tools
- INTRODUCING A TOOL INTO AN ORGANIZATION
Using a testing tool for the first time will not be the best use of the tool either. The skills of a tester are not the same as the skills of the user of the tool.
1 State the main principles of introducing a tool into an organization
Test management tools can provide a lot of useful information, but the information generated by the tool may not be in the form that will be most effective in your context. If you adopt a test management tool when your own testing processes are immature, one option is to follow the standards and processes implied by the way the tool works.
2 State the goals of a proof-of-concept or piloting phase for tool evalua
A report produced by a test management tool (either directly or indirectly through another tool or spreadsheet) may be a very useful report in the moment, but may not be useful in three or six months. If the testing process works well manually, a test management tool can help support the process and make it more efficient.
3 Recognize that factors other than simply acquiring a tool are required
Main principles
The best approach is to define your own processes, taking into account the tool you will be using, and then customize the tool to give you the greatest benefit. However, be aware that the tool should not take the lead, but should support what your organization defines.
Pilot project
Of course, we can sometimes improve our processes in parallel with the introduction of a tool to support those practices, and we can get some good ideas for improvement from how the tools work.
Success factors
From Section 6.2, you should be able to summarize the potential benefits and potential risks of tool support for testing in general. You should recognize that some tools have special considerations, including test execution tools, performance testing tools, static analysis tools, and test management tools.
MOCK EXAM
What is a key
An exhaustive test suite would include
Which statement about testing is true?
For a test procedure that is checking modifications of customers on a
Consider the following list of either product or project risks
- Consider the following list of test process activities
- Test objectives vary between projects and so must be stated in the test
- Which test activities are supported by test data preparation tools?
- Gold card holder who gets upgraded to
- Non-gold card holder who stays in economy
- A person who is bumped from the flight
- Consider the following types of tools
- What is a test condition?
- Which of the following is the most important difference between the metrics-
- If the temperature falls below 18 degrees, the heating is switched
- Which of these
- What is the purpose of confirmation testing?
- Which success factors are required for good tool support within an
- Which of the following best describes integration testing?
- According to the ISTQB Glossary, debugging
- Which of the following could be a root cause of a defect in financial
- Assume postal rates for 'light letters' are
- Consider the following decision table
- What is exploratory testing?
- What does it mean if a set of tests has achieved 90% statement coverage?
- A test plan is written specifically to describe a level of testing
Which of the following important aspects of a good incident report is missing from this incident report. Which of the following tools are most likely to be used by developers?
