NSRL Frequently Asked Questions

Before shipping, we make each release available to vendors of leading forensic tools for testing against their products. For completeness, we also test the data in house against some of the major forensic tools. We try to resolve some questions, but in our experience the vendor support staff will likely give you a better response.

Each release of the NSRL hash set is made available for download free of charge one month after the subscriber CD release. The download is available on this page. You should also verify that the hash set is correct by using the digital signature associated with the downloaded file.

As of the March 2018 RDS release (2.60) the NSRL is no longer accepting new subscriptions for physical DVD releases of our dataset. Our most recent release is available from the NSRL Download page, under Current RDS Hash Set.

Please direct any questions to the NSRL team at nsrl [at] nist.gov (nsrl[at]nist[dot]gov).

The format of the data is described in our paper Data Formats of the NSRL Reference Data Set (RDS) Distribution (PDF).

Yes. The voting RDS files are located here. There is also a separate NSRL voting FAQ.

You should verify the downloads are correct by using the digital signatures for those files.

We purchase most of the software in the NSRL. We try to get everything on major retailers top selling lists. Some software we hear about by word of mouth, some by schedule (like tax programs each tax year, security, antivirus) and some by requests from law enforcement and other agencies. We accept donations from manufacturers and have paperwork to state we will not use the software license . All donations of new software should be COTS shrink-wrapped and exactly what a consumer would purchase. We accept donations of used software as long as it is in useable condition but there is no guarantee that it will make it into the NSRL RDS. We do keep a limited number of duplicate software for media degradation testing and in order to keep a back up of the most popular software, such as operating system packages.

To donate software to the NSRL, please mail packages to:

Department of Commerce - NIST
ATTN: NSRL Project
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970

We apologize, but we cannot lend out copies of the software in our collection.

We make the hashes (MD5, SHA, etc.) available to everyone, but the software itself is (a) stored in an evidence locker, (b) is often donated by vendors with a non-use agreement, or (c) can't be redistributed due to copyright.

However, our experience suggests you might want to try hitting tech swap meets, used bookstores, bargain bins in non-chain stores - they've been a gold mine for us.

We will do what we can within the bounds of the licensing of contents of our collection.

Please contact our subscription department.

No. The NSRL is prevented by law from handling such files, and NSRL policy prevents us from including the hash of a file in the NSRL RDS unless we possess the original copy of that file.

The NDIC HashKeeper project is one source of illicit data hashes (see below).

NIST also has a catalog of digital forensics databases which you may find useful.

The NSRL RDS and the NDIC's Hashkeeper are collections of File Identification Information (FII) which are typically used to identify computer files during forensic investigations of computer systems. The principal differences between the two collections are as follows:

1. Provenance. All NSRL data is derived from purchased or donated software which is retained in a secure facility at NIST. NSRL FII is thus traceable to the original software. Hashkeeper is a central repository of FII donated by various sources, usually obtained by law enforcement during the course of forensic investigations of suspect systems. Hashkeeper FII is not traceable to its source.
2. Court worthiness. The FII published by the NSRL is designed to be admissible as evidence in a U.S. court of law: NSRL data is traceable and verifiable back to the original software packages which are maintained under evidence locker conditions. The Hashkeeper collection is designed to be of use to forensic investigators but is not generally admissible as evidence in a court of law.
3. Scope. NSRL data is strictly limited to that which can be traced to physical installation media which can be obtained and held as evidence at NIST. Hashkeeper data has a potentially broader scope, dependent only on the nature and quantity of FII that law enforcement officials decide to donate.
4. Illicit file data. The NSRL is prohibited by law from obtaining and storing certain forms of illicit file data (e.g. child pornography images) and contains no FII on such data. The Hashkeeper collection is known to contain FII on illicit file data.
5. Specificity of data. The NSRL uses five measures to identify each file (the SHA-1 cryptographic hash, the MD5 cryptographic hash, the CRC32 checksum, the file name and the file size), whereas Hashkeeper supplies only the file name and the MD5 cryptographic hash. There is thus a lower level of confidence in the Hashkeeper data.

Yes, we will be collecting SHA-256, Whirlpool, and several other pieces of metadata that we don't gather now.

The additional metadata will be included in a separate product - the RDS will continue in its present format.

The members of our steering committee (federal, state and local law enforcement) consider the files in the NSRL database as "known" - NOT "known good" OR "known bad" - just "known application files."

NIST does not make a decision about "known bad' or "malicious" or "notable", because there are various case scenarios where that classification is not cut-and-dried.

Note, however, that the NSRL database does contain hashes of files from applications which are traditionally viewed as malicious (encryption, steganography, hacker tools).

You can partition the applications according to your specific needs using the "ApplicationType" field in the "NSRLProd.txt" file - if you consider steganography apps as bad, you can identify them as such using that data.

We have had reports from several investigators that a small number of files - on the order of 10 or 12 - will cause "alerts."

It is our opinion that someone unknown to us has designated all of the file hashes associated with some NSRL hacker applications as "notable" or "malicious" (probably inside a tool that imports the NSRL hash set). Unfortunately, a few of the files used by those hacker apps are very common files used by normally harmless software. If you have a small number of "alert" hits, it is very likely that those are false positives.

Yes, installed software results are included in the RDS. The file entries are marked with the SpecialCode "D".

You can look in a file called NSRLProd.txt and find a column called "ApplicationType". We have classified the programs, and you can look for the description of your interest - steganography, keylogger, office suite, etc.

No. However the NSRL has a research computing environment containing millions of unique original files, along with a database containing metadata about the files (filename, bytesize, etc).

The format for running an algorithm against the file collection is basically that you would submit a job - in the form of your code - to the NSRL. We would then run your job against the file collection, returning the results and your code to you upon completion.

There are various conditions of access to the research environment, including:

• NSRL staff will review the code prior to agreeing to run it
• the research environment is not connected to any external network
• the original files cannot be modified
• no part of the original files can be copied, e.g. exported into the result set
• the result set generated by your job may be made public

Please contact us for details. 

Below is a high-level illustration of the NSRL:

A High-level Illustration of the NSRL