NIST Software Tool Improves Your Doctor’s Vaccination Advice

Behind the scenes at your doctor’s office, there’s a complicated set of information that your providers have to absorb before telling you which vaccinations to get and when. A software tool created at the National Institute of Standards and Technology (NIST) is helping them make better decisions.

The software tool — called the Forecasting for Immunization Test Suite, or FITS — is helping ensure that your doctors are getting correct and up-to-date recommendations about when patients should get their vaccines. While your doctor remains responsible for the final decision, computers are vital resources in the process, as immunization schedules change in response to new medical research and providers must work to stay abreast of it. The FITS tool allows state health care systems to test these computers to find out whether they are providing valid answers for a patient’s circumstances. 

“Children get more vaccinations these days and get them more often,” said NIST computer scientist Mike Indovina. “As medical knowledge is constantly being updated, the schedules become a moving target. It’s difficult for doctors to keep up and know which vaccines a patient should get next — especially if life happens and, for example, their patient misses a visit.”

To help doctors keep up, state health care systems use computerized Immunization Information Systems (IIS), which not only keep track of patients’ immunization records but also have software that recommends future vaccinations. It’s no longer as easy as following a schedule, because the software gets modified several times a year as researchers gather new data about vaccines. 

“They might find out that a particular disease is on the rise and that changes to the vaccination schedules are needed, such as with the hep-A vaccine this year due to a recent spike in hepatitis A cases. Or a brand-new vaccine formula for an illness might come out,” Indovina said. “They’re constantly tweaking the software for these kinds of reasons. It could change the timing for an existing vaccine, or it could change which formulation you receive.”

The FITS tool, which NIST developed with the assistance of the Centers for Disease Control and Prevention (CDC), puts IIS software through its paces by executing more than 800 test cases the CDC created. Each test case — which concerns a single person’s situation and vaccination schedule — should generate a specific immunization recommendation, or “forecast,” by the IIS. FITS measures how closely these forecasts align with the recommendations and standards created by the medical community. FITS also automates the testing process and allows the test cases to be developed, maintained and shared nationwide.

Most recently, FITS has been used by the American Immunization Registry Association (AIRA) to measure the correctness of immunization recommendation services across a large subset of the nation’s IIS. The improvement in IIS performance has been noticeable even over a short period of time. 

“FITS testing has led to the discovery of a number of deficiencies in IIS forecasts,” Indovina said. “This discovery has already led to software corrections and improvements in state IIS immunization services. From the second quarter of 2019 to the third quarter of 2020, 56% of the systems AIRA tested improved by their alignment with the standard test cases.”

FITS performance will have a direct impact on the nation’s response to the current pandemic. AIRA is planning to use FITS for testing COVID-19 vaccine support now that vaccines have become available. 

FITS will also help providers address another problem that has grown since the pandemic struck: Children are falling behind on their recommended vaccinations. The CDC has reported that vaccine rates stalled worldwide last year, and already the numbers for vaccine-preventable diseases such as measles have begun to soar.  Experts fear that the effect of the coronavirus pandemic this year could bring more bad numbers.

“Catch-up schedules are going to be of prime importance in the coming years, as parents start trying to get their kids caught back up on their vaccines,” Indovina said. “The IIS software will provide guidance on how to do that, and it’ll be more important than ever that it give correct and consistent guidance across all systems. FITS will be used to ensure they are recommending those catch-up doses appropriately.”