SAMATE

Welcome to the Software Assurance Metrics And Tool Evaluation (SAMATE) Website!

Software assurance is a set of methods and processes to prevent, mitigate or remove weaknesses and vulnerabilities and ensure that software functions as intended. The Introduction to SAMATE page provides more details. SAMATE’s major efforts include defining bug classes, collecting a corpus of programs with known bugs, and enabling better understanding of tool effectiveness.

The Software Assurance Reference Dataset (SARD) is a growing collection of thousands of test programs with documented weaknesses. The Acknowledgments and Test Suites Descriptions page describes the content. The Manual explains how to use the SARD website.

The Static Analysis Tool Exposition (SATE) is a recurring study designed to advance research in static analysis tools that find security-relevant weaknesses in source code. We provide a set of programs to tool makers, then they run their tools and return tool outputs for analysis.

The Bugs Framework (BF) is a structured, complete, orthogonal, and language-independent classification of software weaknesses (bugs). BF allows unambiguous descriptions of software vulnerabilities.

The AI Bug Finder is a modular and expandable test bed for evaluating AI-based methods for finding bugs in source code.

News:

• Juliet Test Suite for C# version 1.3, developed by NSA Center for Assured Software, was released in August 2020.
• BF was enhanced with the Memory Bugs Model and the definitions of memory related bugs in May 2020.
• SATE VI Ockham Sound Analysis Criteria, NIST Internal Report (IR) 8304 was released in April 2020.

We invite you to participate in the SAMATE mailing list!  The mailing list web site is https://list.nist.gov/samate-discuss. Note that as of November 2019, the SAMATE mailing list moved to Google Groups.

We pronounce SAMATE as suh-mate, which rhymes with date.

If you are looking for the (similarly named) Software Engineering Method And Theory (SEMAT) project web site, please visit http://semat.org/.

This web site was created in July 2005.