An Investigation of Applications of Neural Style Transfer to Forensic Footwear Comparison

Published

February 13, 2023

Author(s)

Gregory Stock

Abstract

Neural Style Transfer (NST) is an application of neural networks that allows one to create an image that has the content of one image and the style of another image. For example, NST can be used to create an image of a cat in the style of Van Gogh's \emphStarry Night}. This study asks two distinct questions. Can we use NST to manufacture pseudo crime scene images of shoe impressions by transferring the style of a crime scene image to a clean impression of a shoe? The idea is that the known, clean image will be transformed to take on the characteristics of a crime scene image, thereby providing a mechanism for researchers to generate pseudo crime scene images that can be used, for example, in training impression recognition algorithms. This is useful because there is a shortage of ground truth known crime scene-like data. We report that our NST procedure does not produce extremely convincing pseudo impressions but the results are nonetheless promising. The second question asks if we can improve crime scene/suspect shoe impression comparison by using NST as a preprocessing, sharpening step. The idea is that the "style" of well-defined edges of the clean impressions will be transferred to the crime scene image. We have a lineup of clean shoe impressions containing the impression of the shoe that created the crime scene image and several other impressions of shoes of the same make, model, and size. We compare the crime scene image to each of the shoes in the lineup using an algorithm that computes similarity scores. The matching shoe should give the highest score with high probability. Next, we combine the crime scene image with an aggregate image of all the clean shoe impressions in the lineup using NST. Then we repeat the comparison process with the NST image instead of the original crime scene image. We observe that, in half of the cases we studied, when we compare the NST image to each of the shoes in the lineup, we can more readily distinguish between the matching shoe and the non-matching shoes than when we do the comparisons with the original crime scene image.

Citation

Grant/Contract Reports (NISTGCR) - NIST GCR 23-040

Report Number

NIST GCR 23-040

NIST Pub Series

Grant/Contract Reports (NISTGCR)

Pub Type

NIST Pubs

Download Paper

https://doi.org/10.6028/NIST.GCR.23-040

Local Download

Keywords

Convolutional neural network, Footwear comparison, Forensic analysis, Neural style transfer, Similarity metrics.

Citation

Stock, G. (2023), An Investigation of Applications of Neural Style Transfer to Forensic Footwear Comparison, Grant/Contract Reports (NISTGCR), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/NIST.GCR.23-040, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=936328 (Accessed December 3, 2024)