7 Essential Principles Of Software Testing

Few can argue against the need for quality control when developing software. Late delivery or software defects can damage a brand’s reputation — leading to frustrated and lost customers. In extreme cases, a bug or defect can degrade interconnected systems or cause serious malfunctions.

The universal law is that success comes from adhering to a set of principles. Whether you want to stay fit, or meet work-related goals, or achieve life’s any specific goals, there are often specific targets and rules we must follow to accomplish them. The same holds true for software testing. As a whole, software testing principles describe how software testers or testing engineers should create bug-free, clear, and maintainable code.

Though testing itself costs money, companies can save millions per year in development and support if they have a good testing technique and QA processes in place. Early software testing uncovers problems before a product goes to market. The sooner development teams receive test feedback, the sooner they can address issues.

To help the software testers to come up with their best software testing strategy, the ISTQB (International Software Testing Qualifications Board) has given seven principles of software testing. These principles of software testing offer general guidelines common for all testing.

According to the ISTQB, here are the common seven testing principles that are widely practiced in the software industry during different phases of software development lifecycle:

1. Exhaustive testing is impossible
2. Early testing
3. Defect clustering
4. Pesticide paradox
5. Testing is context dependent
6. Absence-of-errors fallacy

7 Essential Principles of Software Testing:

Now, what are these principles of software testing? Let’s take a look at these seven Software Testing Principles thoroughly.

1. Exhaustive testing is not possible:

• We have seen that testing helps us find defects and improve software quality. Then one question arises how much testing should we do?
• Let’s say you have a choice: a) Test everything b) Test Nothing c) Test some of the software.
• Well, now your immediate response to that may be to say “Everything must be tested i.e. option a.
• Isn’t it? This implies that we must exercise every aspect of a software system during testing. Now practically what you need to consider is whether we must or even can test completely.
• For example, let’s consider a scenario where we have to test one-digit numeric fields completely.
• So there are 10 numeric values (0 to 9), 26 upper cases (A to Z), 26 upper cases (a to z), 6 special characters, a space, and punctations. So only considering the valid values and ignoring all invalid values we have to test 68 test at least for this example of a one-digit field.
• Now imagine in a real-time scenario, the systems have more than one input field with the fields being of varying sizes. So the problem just gets worse as we look at more realistic examples.
• Testing everything, all combinations of inputs and preconditions is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts.
• Deciding how much testing is enough should take account of the level of risk, including technical and business risks related to the product and project constraints such as time and budget using our business and domain knowledge to test efficiently.

2. Defect Clustering:

• According to ISTQB, this testing principle states that “A small number of modules contain most of the defects discovered during pre-release testing or show the most operational failures.”
• The Pareto Principle states that 80% of software issues come from 20% of modules. On the other hand, the remaining 20% of issues come from 80% of modules. This can happen because

An area of the code is particularly complex and tricky or some cases changing the software and other products tends to cause knock-on defects. Hence, focusing on testing the 20% of modules that create 80% of issues takes priority.

• Reviewing defects and failures in order to improve processes allows us to improve our testing and our requirements, design and development processes.
• For an effective testing strategy, it is necessary to thoroughly examine these areas of the software. The defect clustering method relies on the teams’ knowledge and experience to identify which modules to test. You can identify such risky modules from your experience. Therefore, the team only has to focus on those “sensitive” areas, saving both time and effort.

3. Pesticide Paradox:

• In crop fields if we repetitively use the same pesticide mix to eradicate insects during farming, over the time the insects develop resistance to the pesticide. Thereby ineffective of pesticides on insects happen hence farmers need to improve or use a different approach towards it. The same rule applies to software testing. Over time, as we improve our whole software development life cycle, if the same set of repetitive tests are conducted, the method will be useless for discovering new defects.
• In software testing, the pesticide paradox principle says that “If same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs.

To overcome this pesticide paradox, the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.”

• As the “hot spots” for bugs get cleaned up we need to move our focus elsewhere, to the next set of risks. New and different tests need to be written to exercise different parts of the software or system to potentially find more defects.
• In order to overcome the Pesticide Paradox, it is imperative to regularly review and update the test cases so that more defects can be found. However, if this process is not followed, and the same tests are repeated over and over again, then eventually there will be no new bugs found, but it doesn’t mean the system is 100 % bug free.
• To make testing more effective, testers must constantly look for ways to improve the existing test methods, one way could be automating the existing manual test cases and run those regularly on the area which are already tested. Automating the test suite using different automation testing tools such as Testsigma helps in saving time and effort required in performing repetitive and resource-heavy tasks that are difficult to perform manually.
• At the same time the testers can focus on the new areas, new modules, or new features of the software system. To test new features of the software or system, new tests must be developed.\

4. Testing shows a presence of defects:

• Remember the first principle? Exhaustive testing is not possible, which means we cannot test everything. When 100% testing is not possible isn’t it obvious there would be room for some unidentified bug?
• Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.
• We cannot claim with absolute certainty that a software application is 100% bug-free even if a tester with superb testing skills has tested the application. Testing reduces the number of defects. Even repetitive testing and bug fixes may lead to some more unfound bugs.
• Also, what if the engineering team has developed and tested the software product and made it 99% bug-free but when it is delivered to the customer it does not serve their purpose, isn’t it again the bug system that we need to rework again.?
• Sometimes your application may seem defect-free after several rounds of testing. But when you deploy it into the different environment such as production, there is a possibility that unexpected errors can occur. As a result, the testing team should take care of this and make necessary efforts.

5. Absence of Error — fallacy:

There is another principle we must consider, the customers for software i.e. the people and organizations who buy and use it to aid in their day to day tasks are not interested in defects or numbers of defects, They only concerned about the defects only when, they are directly affected by the instability of the software.

The people using software are more interested in software supporting them in completing their tasks more efficiently and effectively. The software must meet their needs and requirements.

Hence, the absence of Error is a Fallacy i.e. finding and fixing defects doesn’t help if the system built is unusable and does not fulfill the user’s needs and expectations.

6. Early Testing:

• The main objective of software testing is not only about finding defects or bugs, rather it is also about gaining confidence in and providing information about the level of quality. It is also about preventing defects. “If we want to be serious about quality, it is time to get tired of finding bugs and start preventing their happening in the first place.” — Alan Page.
• Hence, testing activities should start as early as possible in the software development life cycle and should be focused on defined objectives.
• Early testing enables quick and timely continuous feedback loops.
• A few examples of early testing could be, testing requirements before coding has started, such as amending issues at the requirement analysis phase is a lot easier and cheaper than doing so right at the end of the product’s lifecycle by which time, the whole areas of functionality might need to be re-written, leading to overruns and missed deadlines.
• The design improvements or code optimization raises the overall development productivity. If the developers receive feedback on the mistakes early, minor bugs don’t have a chance to turn into broad.
• Early testing is necessary to define inaccuracies, contradictions, mismatches, and redundant requirements.
• Bugs detected at the later stage of the software development life cycle are many times more expensive to fix.

7. Testing is context dependent:

• These days, almost every individual is aware of software systems. And all software systems carry different levels of risk. As we discussed earlier in this blog, in the first principle that exhaustive testing is not possible, we need to consider various aspects for coming up with a testing plan such as the level of risk the software system carries.
• Hence, we need to consider how likely it is that the problem or bug will occur and the impact if it happens.
• Some of the bugs we encountered when using software are quite trivial, but others can be costly and damaging — with loss of money, time, or business reputation and even may result in injury or death.
• For example, let’s take a scenario where a user interface has typographical defects.

Does this matter? It may be trivial, but it could have a significant effect, depending on the website and the defect:

• Let’s take 3 scenarios to understand this principle:

Case 1: If my personal family-tree website has my maternal grandmother’s maiden name spelt wrong, my mother gets annoyed and I have to put up with some family teasing, but I can fix it easily and only the family see it (probably).

Case 2: If the company website has some spelling mistakes in the text, potential customers may be put off the company as it looks unprofessional.

Case 3: If a software program miscalculates pesticide application quantities, the effect could be very significant: suppose a decimal point is wrongly placed so that the application rate is 10 times too large. The farmer or gardener uses more pesticide than needed, which raises his costs, has environmental impacts on wildlife and water supplies and has health and safety impact for the farmer, gardener, family and workforce, livestock and pets. There may also be consequent loss of trust in and business for the company and possible legal costs and fines for causing the environmental and health problems.

• In the above 3 cases we saw, how the same typographical defects, can cause trivial to very big issues for an organization. Hence we can conclude as, this principle says: Testing is done differently in different contexts.

Bottom Line

• The biggest achievement of a software development team, you can consider as the positive feedback from end-users, who managed to solve some of their problems with your application or product.
• To get such a result, right after the project initiation, the team has to think through all the activities helpful in making the software product meet and correspond to user expectations.
• To achieve that, your QA team has to perform the testing activities throughout every software development lifecycle or stage, perform verification and validation, use all the advantages of static testing, execute all the necessary techniques of dynamic testing, and maintain effective communication during the project course.
• The earlier you start testing the product and its development processes, the more polished and accomplished it appears in front of the end-users. The fewer bugs users spot in the application and the higher is the trust in the company that has delivered it. And if the product works as planned and meets the user expectations, that means, the real success of the whole team and the whole company that worked on it.
• Principles allow you to live and work to your best while enabling others to understand the best way to interact with you. Similarly, I hope understanding these seven principles of software testing and applying these in real time, will help you as a tester to become more focused and enhance overall testing strategies.