External RNA Controls Consortium

While early gene expression measurements with DNA microarrays were groundbreaking in their ability to reveal biological activity, the results were irreconcilable and irreproducible. Industry leaders approached the National Institute of Standards and Technology (NIST) in 2003 for help with addressing this problem.

NIST hosted the Industry-initiated External RNA Controls Consortium (ERCC) to build the measurement assurance tools needed to support reproducible gene expression measurements. ERCC partners from industry, government, and academia develop RNA spike-in controls and establish analytical methods for bringing reproducible gene expression measurements into routine, high-quality practice. ERCC metrology products include Standard Reference Material (SRM) 2374 for use in the production of RNA spike-in control mixtures, documentary standard CLSI MM16-A, and the erccdashboard R package, an innovative spike-in control analysis software tool. ERCC presentations are hosted on slideshare, as well as the 2014 workshop report.

ERCC control mixture products, derived from SRM 2374, are available through commercial sources. By adding mixtures of ERCC control molecules to experimental samples, scientists are now able to evaluate the technical performance of gene expression experiments. Scientists use ERCC-derived performance metrics for this evaluation, which can be produced using the companion erccdashboard analysis software.

The scientific community has embraced the use of ERCC products, leveraging these tools for measurement assurance, method development, and technology innovation. Gene expression assay and technology developers have incorporated ERCC controls derived from SRM 2374 and the erccdashboard software into their products and protocols.

ERCC Dashboard

The erccdashboard is the first software tool of its kind designed to provide a standard turn-key solution for evaluating the technical performance of any gene expression experiment. This unique tool produces performance metrics that are independent from the type of measurement technology used for an experiment. The erccdashboard gives scientists the ability to gauge the performance of experimental methods, evaluate repeatability and reproducibility of experiments over time and between laboratories, and establish trustworthy experimental results.

Some key performance metrics provided by the erccdashboard for gene expression experiments include:

• Dynamic range;
• Diagnostic performance;
• Limit of detection of ratios;
• Ratio measurement technical variability; and
• Ratio measurement bias, including mRNA fraction differences between experimental samples.

The erccdashboard open source software is available for download here. The dashboard tool is written in the R statistical language, and can be easily incorporated into other analysis software packages.

In a Nature Communications paper published in September 2014, the authors describe the development and demonstration of the erccdashboard analysis methods. 

The stable release version of the erccdashboard , which is recommended for use, is available through the Bioconductor repository.

Development Versions of the erccdashboard code and archives are available on the NIST GitHub repository. 

Selected publications using ERCCs

• Evaluation of the External RNA Controls Consortium (ERCC) reference material using a modified Latin square design
  P. Scott Pine et al., BMC Biotechnology, 2016
• External RNA Controls Consortium Beta Version Update 
  Hangnoh Lee et. al., Journal of Genomics, 2016
• Genomic surveillance elucidates Ebola virus origin and transmission during the 2014 outbreak
  Gire et al., Science, 2014
• Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures
  Munro et. al., Nature Communications, 2014
• Quality control on the frontier
  Paszkiewicz et al., Frontiers in Genetics, 2014
• Reconstructing lineage hierarchies of the distal lung epithelium using single-cell RNA-seq
  Treutlein et. al., Nature, 2014
• Quantitative single-cell RNA-seq with unique molecular identifiers
  Islam et. al., Nature Methods, 2014
• Validation of noise models for single-cell transcriptomics
  Grün et. al., Nature Methods, 2014
• Massively parallel single-cell RNA-seq for marker-free decomposition of tissues into cell types
  Jaitin et. al., Science, 2014
• Quantitative assessment of single-cell RNA-sequencing methods
  Wu et. al., Nature Methods, 2013
• Molecular indexing enables quantitative targeted RNA sequencing and reveals poor efficiencies in standard library preparations
  Fu et. al., Proceedings of the National Academy of Sciences U.S.A., 2013
• Characterization of in vitro transcription amplification linearity and variability in the low copy number regime using External RNA Control Consortium (ERCC) spike-ins
  Kralj and Salit, Analytical and Bioanalytical Chemistry, 2013
• Revisiting global gene expression analysis
  Lovén et. al., Cell, 2012
• Synthetic Spike-in Standards Improve Run-Specific Systematic Error Analysis for DNA and RNA Sequencing
  Zook et. al., PLoS ONE, 2012
• Synthetic spike-in standards for RNA-seq experiments
  Jiang et. al. Genome Research, 2011
• Evaluation of external RNA controls for the standardisation of gene expression biomarker measurements
  Devonshire et. al. BMC Genomics, 2010
• Exploring the use of internal and external controls for assessing microarray technical performance
  Lippa et. al., BMC Research Notes, 2010
• Standards in gene expression microarray experiments
  Salit, Methods in Enzymology, 2006
• The External RNA Controls Consortium: a progress report
  Baker et. al., Nature Methods, 2005
• Proposed methods for testing and selecting the ERCC external RNA controls
  External RNA Controls Consortium, BMC Genomics, 2005