Are Reported Likelihood Ratios Well Calibrated?

Published

October 8, 2019

Author(s)

Peter Vallone, Sarah Riman, Jan Hannig

Abstract

In this work we introduce a new statistical methodology for empirically examining the validity of model-based Likelihood Ratio (LR) systems by applying a general statistical inference approach called generalized fiducial inference [1]. LR systems are gaining widespread acceptance in many forensic disciplines, especially in the interpretation of DNA evidence in the form of probabilistic genotyping systems (PGS). These systems output a Bayes factor, commonly referred to as likelihood ratios in forensic science applications. Methods for examining the validity of such systems is a topic of ongoing interest [2], [3]. In addition to summarizing existing approaches and developing our new approach, we illustrate the methods using the PROVEDIt dataset [4] by examining LR values calculated with open source PG software. [1] Hannig, J., Iyer, H., Lai, R.C.S. and Lee, T.C.M. Generalized Fiducial Inference: A Review and New Results, Journal of the American Statistical Association, 2016, Vol. 111 (515). [2] Brummer, N. Proc. Odyssey 2004 Speaker and Language recognition workshop. ISCA, June 2004, pp. 33–40. [3] Ramos, D. and Gonzalez-Rodriguez J. Forensic Sci Int. 2013 Jul 10;230(1-3):156-69. [4] Alfonse L.E., Garrett A.D., Lun D.S., Duffy K.R., and Grgicak C.M. Forensic Sci. Int. Genet. 2018; 32: pp. 62-70

Citation

Forensic Science International: Genetics Supplement Series

Volume

7

Pub Type

Journals

Download Paper

https://doi.org/10.1016/j.fsigss.2019.10.094

Local Download

Keywords

Likelihood Ratio, Calibration, Weight of Evidence

Citation

Vallone, P. , Riman, S. and Hannig, J. (2019), Are Reported Likelihood Ratios Well Calibrated?, Forensic Science International: Genetics Supplement Series, [online], https://doi.org/10.1016/j.fsigss.2019.10.094, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=929077 (Accessed October 9, 2025)