ACMD Seminar: Modeling AI Supply Chains: How Increased Outsourcing and Complexity in AI/ML Development Introduces New Considerations for Machine Learning

Aspen Hopkins
Ph.D. Candidate, Dept. of Electrical Engineering & Computer Science, Massachusetts Institute of Technology

Tuesday, Feb. 25, 2025, 3:00-4:00 PM ET (1:00-2:00 PM MT)

A video of this talk will be made available to NIST staff in the Math channel on NISTube, which is accessible from the NIST internal home page. It will be taken down from NISTube after 12 months at which point it can be requested by emailing the ACMD Seminar Chair.

Abstract: As machine learning (ML) and artificial intelligence (AI) systems grow increasingly complex and ubiquitous, researchers and developers have begun to outsource aspects of their development. Rather than develop fully "in house", deployed AI/ML systems may now be the result of multiple entities contributing data (synthetic, augmented, or real), models, training, evaluations, or other resources or services. These networks of AI components–or AI supply chains–grow increasingly complex, and introduce new considerations for building, deploying, and evaluating AI. 

In this talk, we will introduce a model of these "AI supply chains" as a directed graph. Grounded in this model, we illustrate how AI supply chains complicate the achievability of key machine learning objectives through two case studies on explainability and fairness. Specifically, we will show theoretical and empirical evidence that error in local linear explanations increases with an AI supply chain’s width and depth, and how fairness (e.g., conditional independence) applied to upstream models limits the expressiveness of downstream fine-tuned models. We will discuss how specific characteristics of AI supply chains result in these outcomes, and outline on-going work exploring the implications of data deletions and hidden data interactions across these networks.

Bio: Aspen K. Hopkins is a Ph.D. candidate advised by Aleksander Madry in MIT’s EECS program. Her research focuses on fundamental aspects of model development, with particular emphasis on AI supply chains—the increasingly fragmented and outsourced networks through which many modern AI systems are developed. Unlike traditional, siloed ML pipelines, AI supply chains involve multiple participants contributing models, data, and computing resources, introducing interdependencies, feedback loops, and additional complexity to modeling efforts. Beyond AI supply chains, Aspen is interested in mechanistic interpretability and the intersection of human-computer interaction (HCI) and ML. Aspen’s research is funded in part by a Siebel Scholarship and MIT Schwarzman College of Computing’s Social and Ethical Responsibilities of Computing award. She has previously worked at Apple, NASA JPL, and in venture/startups.

Host: Allison Carson

Note: This talk will be recorded to provide access to NIST staff and associates who could not be present to the time of the seminar. The recording will be made available in the Math channel on NISTube, which is accessible only on the NIST internal network. This recording could be released to the public through a Freedom of Information Act (FOIA) request. Do not discuss or visually present any sensitive (CUI/PII/BII) material. Ensure that no inappropriate material or any minors are contained within the background of any recording. (To facilitate this, we request that cameras of attendees are muted except when asking questions.)

*Safety Precaution: The hallway leading from the Courtyard to the exit closest to B-111 and B-113 will be used by contractors to move debris, machinery, and other supplies, as well as will be heavily trafficked by the contractors throughout the process. Be aware of the safety precautions posted during this time.

Note: Visitors from outside NIST must contact Meliza Lane at least 24 hours in advance.