NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.

Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.

U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Search Publications by: Raghu N Kacker (Fed)

Search Title, Abstract, Conference, Citation, Keyword or Author
Displaying 1 - 25 of 194

Data Frequency Coverage Impact on AI Performance

April 15, 2025
Author(s)
Erin Lanus, Brian Lee, Jaganmohan Chandrasekaran, Laura Freeman, M S Raunak, Raghu Kacker, David Kuhn
Artificial Intelligence (AI) models use statistical learning over data to solve complex problems for which straightforward rules or algorithms may be difficult or impossible to design; however, a side effect is that models that are complex enough to

Proxima: A Proxy Model-Based Approach to Influence Analysis

September 25, 2024
Author(s)
Sunny Shree, Yu Lei, Raghu Kacker, David Kuhn
Machine learning (ML)-based Artificial Intelligence (AI) systems rely on training data to perform optimally, but the internal workings of how ML models learn from and use this data are often a black- box. Influence analysis provides valuable insights into

Assessing the Degree of Feature Interactions that Determine a Model Prediction

September 17, 2024
Author(s)
Krishna Khadka, Sunny Shree, Yu Lei, Raghu Kacker, David Kuhn
Machine Learning (ML) models rely on capturing important feature interactions to generate predictions. This study is focused on validating the hypothesis that model predictions often depend on interactions involving only a few features. This hypothesis is

On Combinatorial Security Testing for the Tor Anonymity Network Client

September 17, 2024
Author(s)
Dimitris Simos, Bernhard Garn, Dominik-Philip Schreiber, Manuel Leithner, David Kuhn, Raghu Kacker
In this paper, we present an application of combinatorial security testing to the well-known anonymity network Tor. Rigorous testing of the Tor network is important to evaluate not only its functionality, but also the security it provides to its users

A unified model of core metrological concepts

July 10, 2024
Author(s)
David W. Flater, Raghu N. Kacker, Douglas Foxvog
The definitions of core metrological terms, especially quantity, quantity value, and unit, have been the subject of years of wrangling in standards organizations. A chronic disagreement exists over whether quantities are conceptualized primarily as

Leveraging Combinatorial Coverage in the Machine Learning Product Lifecycle

June 27, 2024
Author(s)
Jaganmohan Chandrasekaran, erin lanus, tyler cody, laura freeman, Raghu N. Kacker, M S Raunak, D. Richard Kuhn
The data-intensive nature of machine learning (ML)-enabled systems introduces unique challenges in test and evaluation. We present an overview of combinatorial coverage, exploring its applications across the ML-enabled system lifecycle and its potential to

Ensuring Reliability Through Combinatorial Coverage Measures

May 14, 2024
Author(s)
M S Raunak, D. Richard Kuhn, Raghu N. Kacker, Yu Lei
A key question in software assurance is, "How much testing is enough?" Coverage criteria such as statement or branch coverage were developed to help answer this question, but we also need to measure whether tests are sufficiently representative of inputs

Assured Autonomy through Combinatorial Methods

May 2, 2024
Author(s)
David Kuhn, M S Raunak, Raghu Kacker, Jaganmohan Chandrasekaran, Erin Lanus, Tyler Cody, Laura Freeman
Autonomous systems are proliferating rapidly, with strong interest in everything from vacuum cleaners and lawnmowers, to self-driving cars and autonomous farm equipment. Can these systems be trusted to function safely? Many conventional software

Combinatorial testing for building reliable systems

February 5, 2024
Author(s)
M S Raunak, D. Richard Kuhn, Raghu N. Kacker, Yu Lei
Combinatorial testing is an approach where test suites are developed by efficiently covering interactions of parameter values and configuration settings. Multiple studies over the years have shown the interesting phenomenon where almost all defects in a

Ordered t-way Combinations for Testing State-based Systems

May 29, 2023
Author(s)
D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if the code has been exercised sufficiently to

Synthetic Data Generation Using Combinatorial Testing and Variational Autoencoder

May 29, 2023
Author(s)
Krishna Khadka, Jaganmohan Chandrasekaran, Yu Lei, Raghu N. Kacker, D. Richard Kuhn
Data is a crucial component in machine learning. However, many datasets contain sensitive information such as personally identifiable health and financial data. Access to these datasets must be restricted to avoid potential security concerns. Synthetic

Ordered t-way Combinations for Testing State-based Systems

June 13, 2022
Author(s)
D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if code has been exercised sufficiently to ensure

Developing multithreaded techniques and improved constraint handling for the tool CAgen

June 8, 2022
Author(s)
Michael Wagner, Manuel Leithner, Dimitris Simos, D. Richard Kuhn, Raghu N. Kacker
CAgen is a state-of-the-art combinatorial test generation tool that is known for its execution speed. In addition, it supports an extensive list of features such as constraint handling, higher-index arrays, and import and export of models/test sets in

The Path to Consensus on Artificial Intelligence Assurance

March 15, 2022
Author(s)
Laura Freeman, Feras Batarseh, D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Widescale adoption of intelligent algorithms requires that Artificial Intelligence (AI) engineers provide assurances that an algorithm will perform as intended. Providing such guarantees involves quantifying capabilities and the associated risks across

A Pseudo Exhaustive Software Testing Framework for Embedded Digital Devices in Nuclear Power

June 14, 2021
Author(s)
Athira Jayakumar, D. Richard Kuhn, Brandon Simons, Aidan Collins, Smitha Gautham, Richard Hite, Raghu N. Kacker, Abhi Rajagopala, Carl Elks
The major challenge faced by the nuclear industry related to software testing of digital embedded devices is the identification of practical software (SW) testing solutions that provide a strong technical basis and is at the same time effective in

Combinatorially XSSing Web Application Firewalls

May 28, 2021
Author(s)
Bernhard Garn, Daniel S. Lang, Manuel Leithner, D. Richard Kuhn, Raghu N. Kacker, Dimitris Simos
Cross-Site scripting (XSS) is a common class of vulnerabilities in the domain of web applications. As it remains prevalent despite continued efforts by practitioners and researchers, site operators often seek to protect their assets using web application

Combinatorial Testing Metrics for Machine Learning

April 12, 2021
Author(s)
Erin Lanus, Laura Freeman, D. Richard Kuhn, Raghu N. Kacker
This short paper defines a combinatorial coverage metric for comparing machine learning (ML) data sets and proposes the differences between data sets as a function of combinatorial coverage. The paper illustrates its utility for evaluating and predicting

Combinatorial Test Generation for Multiple Input Models with Shared Parameters

March 17, 2021
Author(s)
Chang Rao, Nan Li, Yu Lei, Jin Guo, YaDong Zhang, Raghu N. Kacker, D. Richard Kuhn
Combinatorial testing typically considers a single input model and creates a single test set that achieves t-way coverage. This paper addresses the problem of combinatorial test generation for multiple input models with shared parameters. We formally

Combinatorial Methods for Explainable AI

October 24, 2020
Author(s)
David R. Kuhn, Raghu N. Kacker, Yu Lei, Dimitris Simos
This paper introduces an approach to producing explanations or justifications of decisions made by artificial intelligence and machine learning (AI/ML) systems, using methods derived from fault location in combinatorial testing. We use a conceptually

Vulnerability trends in web servers and browsers

September 21, 2020
Author(s)
M S Raunak, David Kuhn, Richard Kogut, Raghu Kacker
In previous work we have looked at trends in vulnerabilities due to ordinary programming errors [2, 3]. This analysis focuses on two of the most widely used types of software in today's internet, web browsers and web servers. In addition to reports of

Effectiveness of dataset reduction in testing machine learning algorithms

August 25, 2020
Author(s)
Raghu N. Kacker, David R. Kuhn
Abstract— Many machine learning algorithms examine large amounts of data to discover insights from hidden patterns. Testing these algorithms can be expensive and time-consuming. There is a need to speed up the testing process, especially in an agile
Was this page helpful?