Combinatorial testing for building reliable systems

Published

February 5, 2024

Author(s)

M S Raunak, D. Richard Kuhn, Raghu N. Kacker, Yu Lei

Abstract

Combinatorial testing is an approach where test suites are developed by efficiently covering interactions of parameter values and configuration settings. Multiple studies over the years have shown the interesting phenomenon where almost all defects in a system originate from interactions of a few specific parameters or settings. Efficient algorithms compressing these value combinations into a small number of tests have made this method practical for industrial use, providing better testing at lower cost. Through this approach, fault detection close to the level of exhaustive testing can be achieved with a 20x to 700x reduction in the test suite size. Since most defects in systems can be discovered with systematic testing using 2- to 6-way interactions of parameter values, utilizing this approach can help us develop highly reliable systems.

Citation

IEEE Reliability Magazine

Volume

1

Issue

1

Pub Type

Journals

Download Paper

https://doi.org/10.1109/MRL.2024.3355874

Local Download

Keywords

combinatorial testing, reliability

Citation

Raunak, M. , Kuhn, D. , Kacker, R. and Lei, Y. (2024), Combinatorial testing for building reliable systems, IEEE Reliability Magazine, [online], https://doi.org/10.1109/MRL.2024.3355874, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=957313 (Accessed November 21, 2024)