

hypotheses (see hierarchy of testing difficulty below), testing cannot identify all the defects within
software.2 Instead, it furnishes a criticism or comparison that compares the state and behavior of the
product against test oracles—principles or mechanisms by which someone might recognize a problem.
These oracles may include (but are not limited to) specifications, contracts,3 comparable products, past
expectations, relevant standards, applicable laws, or other criteria.
does not function properly under specific conditions.4 The scope of software testing often includes
be used to correct the process by which software is developed.5:41–43
Every software product has a target audience. For example, the audience for video game software is
users, its target audience, its purchasers and other stakeholders. Software testing aids the process of
attempting to make this assessment.
A fundamental problem with software testing is that testing under all combinations of inputs and
preconditions (initial state) is not feasible, even with a simple product.4:17–187 This means that the
number of defects in a software product can be very large and defects that occur infrequently are
difficult to find in testing. More significantly, non-functional dimensions of quality (how it is supposed
to be versus what it is supposed to do)—usability, scalability, performance, compatibility, reliability—can
be highly subjective; something that constitutes sufficient value to one person may be intolerable to
Software developers can’t test everything, but they can use combinatorial test design to identify the
minimum number of tests needed to get the coverage they want. Combinatorial test design enables
users to get greater test coverage with fewer tests. Whether they are looking for speed or test depth,
they can use combinatorial test design methods to build structured variation into their test cases