16. Testing continued.

Test Selection

It is important to realise that with the best will in the world it is not possible to test every part of the system.

Consider a system having 8 data inputs that can applied in any combination. The maths says that there are over 16 million possible combinations of just 8 inputs!

So the skill of the testing team is to carefully select a practical number of tests that can be carried out in the time available and yet be confident that the system is working.

This is why complicated software often has bugs even after being released. It is simply not practical to test everything.

Test Data

It is important that any testing which checks the validation routines should include:

Data that is within the normal range and will be accepted

Data that is on the extreme limits of the range but should be accepted e.g. if the validation says that price <=£100 then £100 should be tested as that is right at the outer limit.

Data that should fail (erroneous data) should be tested. For the test mentioned previously, the test might be: enter £100.01

Iteration or Looping Around.
If an issue is found it may actually be a problem with the user requirement itself. It is quite common for the user requirement to not consider an unexpected situation.

For instance, the document might state that the system is to be used by 10 people at the same time. But by the time testing is underway something changes in the company and now the system needs to deal with 20 people instead.

In which case the client has to agree to change the user requirement document.

Then testing continues with the modified user requirement document.

Changing user requirements part way through a project causes delays and extra cost. It is often the most common reason a project fails especially on large projects.

Testing is usually an 'iterative' process.