Estimating Fault Detection Effectiveness

Published

April 1, 2014

Author(s)

David R. Kuhn, Raghu N. Kacker, Yu Lei

Abstract

[Poster] A t-way covering array can detect t-way faults, however they generally include other combinations beyond t-way as well. For example, a particular test set of all 5-way combinations is shown capable of detecting all seeded faults in a test program, despite the fact that it contains up to 9-way faults. This poster gives an overview of methods for estimating fault detection effectiveness of a test set based on combinatorial coverage for a class of software. Detection effectiveness depends on the distribution of t-way faults, which is not known. However based on past experience one could say for example the fraction of 1-way faults is F(sub)1 = 60%, 2-way faults F(sub)2 = 25% F(sub)3 = 10% and F(sub)4 = 5%. Such information could be used in determining the required strength t. It is shown that the fault detection effectiveness of a test set may be affected significantly by the t-way fault distribution, overall, simple coverage at each level of t, number of values per variable, and minimum t-way coverage. Using these results, we develop practical guidance for testers.

Proceedings Title

Proceedings of the Seventh IEEE International Conference on Software, Testing, Verification and Validation (ICST 2014)

Conference Dates

March 31-April 4, 2014

Conference Location

Cleveland, OH

Conference Title

Third International Workshop on Combinatorial Testing

Pub Type

Conferences

Download Paper

https://doi.org/10.1109/ICSTW.2014.69

Keywords

combinatorial testing, software testing, test coverage

Citation

Kuhn, D. , Kacker, R. and Lei, Y. (2014), Estimating Fault Detection Effectiveness, Proceedings of the Seventh IEEE International Conference on Software, Testing, Verification and Validation (ICST 2014), Cleveland, OH, [online], https://doi.org/10.1109/ICSTW.2014.69 (Accessed December 17, 2024)