Recommended Minimum Standard for Vendor or Developer Verification of Code

Threat modeling

Threat modeling helps identify key or potentially overlooked testing targets.

Section 2.1. Threat modeling methods create an abstraction of the system, profiles of potential attackers and their goals and methods, and a catalog of potential threats. Threat modeling can identify design-level security issues and help focus verification.

Automated testing

As testing is automated, it can be repeated often, for instance upon every commit or before an issue is retired.

Section 2.2. Automated testing can run tests consistently, check results accurately, and minimize the need for human effort and expertise. Automated testing can be integrated into the existing workflow or issue tracking system.

Code-based (static) analysis

Use a code scanner to look for top bugs.

Section 2.3. Static analysis tools can check code for many kinds of vulnerabilities and for compliance with the organization’s coding standards. For multi-threaded or parallel processing software, use a scanner capable of detecting race conditions.

Review for hardcoded secrets. 

Section 2.4. Heuristic tools can be somewhat effective checking for hardcoded passwords and private encryption keys since functions or services taking these as parameters have specific interfaces.

Dynamic analysis (i.e., run the program on test cases)

Run with built-in checks and protections.

Section 2.5. Programming languages, both compiled and interpreted, provide many built-in checks and protections.

Create “black box” test cases.

Section 2.6. “Black box” tests can address functional specifications or requirements, negative tests (invalid inputs and testing what the software should not do), denial of service and overload attempts, input boundary analysis, and input combinations.

Create code-based structural test cases.

Section 2.7. Code-based, or structural, test cases are based on the implementation, that is, the specifics of the code. Code-based test cases may also come from coverage metrics.

Use test cases created to catch previous bugs.

Section 2.8. Test cases which have been created to specifically show the presence (and later, the absence) of a bug can be used to identify issues in the absence of more general “first principles” assurance approaches for detecting bugs.

Run a fuzzer.

Section 2.9. Fuzzers can try an immense number of inputs with minimal human supervision. The tools can be programmed with inputs that often reveal bugs, such as very long or empty inputs and special characters. 

If the software might be connected to the Internet, run a web app scanner.

Section 2.10. If there is a network interface, use a dynamic security testing tool (e.g., web application scanner) to detect vulnerabilities.

Check included software

Use similar techniques to gain assurance that included libraries, packages, services, etc. are no less secure than your code.

Section 2.11. Use the verification techniques recommended in this section to gain assurance that included code is at least as secure as code developed locally. The components of your software must be continually monitored against databases of known vulnerabilities; a new vulnerability in existing code may be reported at any time.

Fix bugs

Fix critical bugs that are uncovered.

Correct critical bugs as soon as possible and make process improvements necessary to prevent such bugs in the future, or to at least catch them earlier in the development process.