Software QA/Tester FAQs: An Example on Reliability Testing?

Software QA/Testing

Glossary and Technical FAQs

Q: What is reliability testing? (Cont'd...)

When we identify a failure, we send the software back to the developers, for repair. The developers build a new version of the software, and then we do another test iteration. We track failure intensity (e.g. failures per transaction, or failures per hour) in order to guide our test process, and to determine the feasibility of the software release, and to determine whether the software meets the customer's reliability requirements.

Q: Give me an example on reliability testing.

A: For example, our products are defibrillators. From direct contact with customers during the requirements gathering phase, our sales team learns that a large hospital wants to purchase defibrillators with the assurance that 99 out of every 100 shocks will be delivered properly.

In this example, the fact that our defibrillator is able to run for 250 hours without any failure, in order to demonstrate the reliability, is irrelevant to these customers. In order to test for reliability we need to translate terminology that is meaningful to the customers into equivalent delivery units, such as the number of shocks. We describe the customer needs in a quantifiable manner, using the customer’s terminology. For example, our of quantified reliability testing goal becomes as follows: Our defibrillator will be considered sufficiently reliable if 10 (or fewer) failures occur from 1,000 shocks.

Then, for example, we use a test/analyze/fix technique, and couple reliability testing with the removal of errors. When we identify a failed delivery of a shock, we send the software back to the developers, for repair. The developers build a new version of the software, and then we deliver another 1,000 shocks into dummy resistor loads.

We track failure intensity (i.e. number of failures per 1,000 shocks) in order to guide our reliability testing, and to determine the feasibility of the software release, and to determine whether the software meets our customers' reliability requirements.

[Continued on next page...]