NIST-Led Study Finds Variations in Quantitative MRI Scanners’ Measurements

Magnetic resonance imaging (MRI) is widely used in medicine to detect, diagnose and treat diseases such as cancer, while relying on experts’ interpretation of images. Quantitative MRI, which obtains numerical measurements during the scans, can now potentially offer greater accuracy, repeatability and speed — but rigorous quality control is needed for it to reach its full potential, according to a new study.

Researchers at the National Institute of Standards and Technology (NIST) led the study by 11 institutions comparing measurements by 27 MRI scanners from three vendors at nine clinical sites around the country. To obtain reference values and disentangle sources of bias and variation, the study used a tissue stand-in, or “phantom,” originally developed at NIST for evaluating the performance of MRI machines and related software.

MRI uses magnetic fields and radio waves to visualize the body’s internal structures, especially soft tissues. Traditional MRI poses several challenges. In addition to subjective image analysis, MRI scanner performance may drift, or different machines may produce contradictory images of the same patient. 

Quantitative MRI offers prospects for more consistent disease detection, diagnosis and treatment without the need for tissue biopsy. Ideally, numerical measurements of tumors and other disease markers could be reproduced across many different patients, scanners and clinics over time — and potentially reduce medical costs. Organizations such as the International Society for Magnetic Resonance in Medicine are promoting quantitative MRI, but it is currently used only in research, not routine patient care. 

The new study compared MRI scanner measurements of a value called T1, a property of water molecules that can depend on the surrounding tissue. T1 of the pixels in images is one of the parameters that could be measured by clinical MRI systems. By contrast, subjective interpretations of MRI images look at “T1-weighted” judgments, which are qualitative and nonnumerical.

“A big difference between the T1 and current T1-weighted is that the T1-weighted isn’t quantitative,” study leader Katy Keenan of NIST said. “The pixel values themselves cannot be compared to pixel values in other datasets. It is not easy to compare T1-weighted data across datasets. In the case of carefully acquired T1, comparison of the pixel values is possible, because they have a quantitative meaning.”

The study found that T1 measurements can be subject to significant bias and variation. There was no consistent pattern of discrepancy between vendors, and as a result, a diagnostic threshold value determined on one MRI system can’t be transferred to other MRI systems. In some cases, these variations could make a clinical difference in diagnosing a benign versus a malignant brain tumor, seriously affecting patient care, the study found.

As a remedy, the study recommended establishing rigorous quality control procedures for quantitative MRI to promote confidence and stability in measurement techniques and to transfer measurement thresholds for diagnosis, disease progression, and treatment monitoring from research venues to the entire clinical community. The study results echo previous findings by other researchers.

Earlier efforts to use T1 values to categorize brain tumors were impeded by technical inconsistencies and found to be unreliable. But recent advances in quantitative measurement methods have led to improvements in accuracy, repeatability and acquisition speed. The new study is a step toward applying diagnostic threshold T1 measurement across multiple clinical sites. 

Paper: K.E Keenan, Z. Gimbutas, A. Dienstfrey, K.F. Stupic, M.A. Boss, S.E. Russek, T.L. Chenevert, P.V. Prasad, J. Guo, W.E. Reddick, K.M. Cecil, A. Shukla-Dave, D.A. Nunez, A.S. Konar, M. Liu, S.R. Jambawalikar, L. Schwartz, J. Zheng, P. Hu and E.F. Jackson. Multi-site, multi-platform comparison of MRI T1 measurement using the system phantom. PLOS One. Published online June 30, 2021. DOI: 10.1371/journal.pone.0252966