PSCR Staff Spotlight: Exploring the Voices of First Responders with the NIST Visualization and Usability Group

The NIST Visualization and Usability Group performs research to develop user-centered measurement and evaluation methods, guidelines, and standards by applying human factors, cognitive science, user-centered design, and usability principles to improve human system interaction. In 2017, some of the Usability team’s experts joined the PSCR team to launch the Voices of First Responders project and provide guidance for the public safety research and development (R&D) community. The project’s objective is to ensure that technologies are developed for and with first responders to achieve their goals with effectiveness, efficiency, and satisfaction in their specified contexts of use.

PSCR’s communications team interviewed Dr. Choong, Dr. Dawkins, and Dr. Buchanan on the Usability Team to learn more about the history of the Voices of First Responders project and what audiences can expect to hear in the “Voices of First Responders: PSCR’s Survey Analyzer Tool in Action” webinar!

Can you describe the purpose of the Usability team and your current roles on the team? 

Yee-Yin Choong (YC): We’re a multi-disciplinary group and our focus is human-centered research. Our vision is “championing the human in information technology.” And the “human” is a loose term – to me, it’s any human that interacts with technology; be it the developers, administrators, policy makers, or end users. In 2017 our team was brought under the PSCR umbrella to focus on the User Interface/User Experience (UI/UX) portfolio to provide our expertise in usability, human-centered design, and human factors, to support their vision. Currently I’m the Usability Lead supporting the UI/UX portfolio.

Shanee Dawkins (SD): Specifically regarding PSCR, the purpose of our team is to look at the humans involved in public safety communications technology, so that’s anyone involved with communicating during incident response. This includes first responders, 9-1-1 call takers and dispatchers, fire fighters, EMS, and law enforcement. We focus on them, their communication technology, their problems, and their needs, so that they can be improved in the future. I have a Ph.D. in computer science where I studied human-centered computing and human-computer interaction. My expertise brings a unique viewpoint to our multidisciplinary team.

Kerrianne Buchanan (KB): I’m the relative newcomer to the team; I started in July 2020. Yee-Yin, Shanee, and others in our group have been working together for a long time on this project; they’ve worked really hard to collect the interview and survey data and I’m here to help them with continuing to analyze the data and disseminate the findings. Our current focus is to help get the Voices of First Responders interview and survey data out into the world so people can see it and use it. We’re focusing on writing reports that make actionable recommendations for industry and researchers, as well as bringing attention to the specific details and findings that support the bigger themes we’ve found in our research.

What brought you to NIST?

YC: My undergraduate and master’s degrees were in electronics engineering. I grew up in Taiwan and came to the U.S. to pursue advanced degrees. But when I was in a PhD program for electronics engineering, I realized that I enjoyed interacting with people more than the hardware. I came across a research discipline called Human Factors in Industrial Engineering at Purdue University, which deals with how people process information and interact with technology. This was perfect for my research interests, so I switched majors.

I then worked in the private sector for ten years focusing on usability engineering in e-commerce, but there was not much opportunity for research. We would make improvements to products, but we weren’t doing research truly from the bottom-up to understand the needs of our users and address them. In 2006, I joined NIST to focus on research and bring positive impacts from a human factors perspective.

SD: I’ve been with NIST almost 10 years this year. My dissertation work studied the usability and accessibility of electronic voting systems, so I came to NIST to work on voting system usability and accessibility standards. I then worked with another group evaluating language translation systems. And now I work with PSCR on the Voices of First Responders project, and also on the usable cybersecurity project, where I run the phishing project looking at humans’ susceptibility to phishing attempts. 

KB: I’m brand new to NIST. I graduated from my doctoral program in 2019 in psychology where I was studying how people interact with other people. After graduating I worked at a contracting company for about a year doing communications research. I heard about NIST through a friend and applied to this job, since I thought it would be a good extension of my background and research skills to begin studying how people interact with technology.

Tell me about your webinar, Voices of First Responders: PSCR’s Survey Analyzer Tool in Action and what you’re excited for the attendees to learn about!

YC: We are really excited about the whole project!  We named this project the “Voices of First Responders” because the research we’re conducting is directly from their voices. We interviewed almost 200 people and received over 7,000 responses from a nationwide survey, with a good sample size including rural and metropolitan first responders covering all four disciplines. We learned a lot by understanding their communication technology usage, what kind of problems they experience, and what they perceive as useful to facilitate their jobs. And now we can slice and dice the data in different ways. Webinar attendees will see that they can use the Survey Analyzer tool to look across disciplines or drill down on one discipline; they can look across devices or at one device; and they can look at different roles such as frontline first responders or chiefs/managers to better understand their perspectives. People will learn how to use the tool to find the information that’s most interesting to them, or data to support the project they’re working on. And then they can view and download charts, and more. I think it will provide a lot of utility for academia, technologists, and government alike. People want to see the concrete data to support their management decisions and research, and we have that to share.

SD: I agree, I want people to be excited about the tool as a whole if they haven’t heard about it over the past two years! We’ve been building on it and making it more exciting, so I hope they’re able to come and learn what we’ve done to this point and use the new interactive part going forward.

KB: I’m really happy we’re doing the webinar because it will give us a chance to walk people through how to use the data. We’ve collected so much data – hours and hours of interviews and thousands of survey responses – so now that we’ve done our analysis, we want to get the data out there for other people to use.

It's my understanding that the Voices of First Responders study is the largest that's ever been fielded with first responders about their tech attitudes and behavior. Can you tell me about how this project started and how it's adapted over the years?

YC: The PSCR team recognized that UI/UX is key for any technology, so we were brought in to help with that aspect of their research. We employed a multi-phase approach with both qualitative and quantitative components to truly provide data-driven insights, so it’s not just based in theory. Our first step was conducting qualitative analysis through the almost 200 interviews with first responders. In the second phase, we performed quantitative analysis to confirm the findings from phase one by conducting a nationwide survey completed by over 7,000 first responders.

We are now in the final phase of the project and the overarching goals have not changed. The project itself has been broadened because originally we were exploring something that has never been done before. We had to start with the exploratory qualitative data because we needed to know the problem space to research before we started collecting confirmatory quantitative data. And in this final phase we’re conducting meta-inference, meaning we are integrating the qualitative and quantitative data to provide further insights and actionable recommendations. We’ll be concluding this five-year project with a presentation at PSCR’s Annual Stakeholder Meeting in June 2022, and publishing a final publication by the end of the fiscal year.

SD: Building on what Yee-Yin said, our goal was to improve communication technology for first responders; but you can’t improve it if you don’t know their experiences. The first step in all of our research was to understand the user, their context of use, and their needs in order to build their requirements. It was a long, multiphase process that took many years.

KB: There’s been a huge, on-going effort from the First Responder Network Authority (FirstNet) and other organizations to help first responders do their jobs better and get them the technology they need. Our focus had been on understanding experiences with technology from first responders themselves. It’s been really interesting that we ended up finding that some non-technical aspects, like the price of devices, have huge implications for what first responders are able to use. So many of our findings make intuitive sense and resonate with first responders’ experiences, and now it’s great to have the empirical data to back up those viewpoints. I hope that our work will provide additional context about what first responders experience and how they see technology. I also hope that our work will be used to develop new technology that will help first responders in the future.

What is one memorable finding or interaction you’ve had with a first responder throughout this project?

YC: During one of the in-person interviews in phase one, I spoke with a fire chief who was in tears telling us how he lost staff due to an air tank that did not meet heat resistance standards. His staff entered a burning building, the air tanks failed, and they ended up breathing in hot air and dying. Some of his words still stick with me. “Shouldn’t this be common sense?” he said, “We are firefighters, how could our air tanks not be heat proof?” Stories like this make me even more passionate about getting the voices of first responders out there. Oftentimes, technology is not designed or developed with them. Maybe it’s developed with them in mind, but not with them, so their requirements are not captured accurately. Hopefully this research has more positive impacts for the community because they are real people risking their lives to serve and protect us.

SD: Over and over during the interviews we heard the same thing from first responders. We would ask them, “What grand thing do you want if you could have anything you need to do your job?” And everyone said they don’t want fancy things necessarily; they really just need what they have to work. They don’t rely on technology, they rely on their senses – what they see, what they hear, and what they’re feeling – so they have technology that supplements that, but they need the technology to work. That’s the bottom line.

KB: Unfortunately, I haven’t had that much one-on-one experience with the first responders other than getting to share our results at conferences and meetings. One aspect of this project that I’m really proud of is that we had a paper accepted into ISCRAM, an international crisis management conference that was focused on the needs of rural first responders. It was really interesting to focus on the rural perspective, and the challenges rural environments bring to first responders. I’m hoping that our work will be able to help mitigate some of the big challenges rural first responders face.

What is the most impactful research you’ve seen, or that you hope to see, come out of the interviews and surveys that were conducted for this effort?

YC: I would like to see R&D projects or the industry emphasize more on user requirements and testing, and make sure they have the intended audience or users for a product testing it and making sure it works in their context and environment. In all of our publications we’ve been recommending designing products and testing products with first responders to make sure the product works for them.

SD: I’m knee deep in data all the time, so I don’t often have a chance to look up. But I do know from going to the PSCR Annual Stakeholder Meeting that every year we hear first responders saying our research is great and everyone needs to hear it; that all of the developers and designers need to incorporate our work into what they’re doing. So that’s very rewarding to hear. I know people are using the website and Survey Analyzer tool and looking at the data and I’m excited to see how that impacts development in the future.

KB: I specifically hope people can dig into the portions of the survey that deal with futuristic devices and technology that might be useful. It would be interesting to see how we can address the problems that a lot of first responders are experiencing with new technology.

What do you think the broader research community or even the industry as a whole should know about The Voices of First Responders project?

YC: I hope our research shows that the industry needs to involve usability, human factors, and user experience experts in their project teams. Bring your target audience into your project life cycle early on so you don’t have to rework your products or service. Through our overall work, we provide a good foundation and resources including data analysis and methodology recommendations, so treat this research like a useful resource for the community. Hopefully they’re as passionate as we are about whatever projects they’re developing, and there are humans in the loop to make sure they’re successful.

SD: I want others to know that we are a neutral party that looked at what first responders need in their communication technology; and the data is freely available and accessible via the Survey Analyzer tool. This tool is for our stakeholders; for anyone researching or designing or developing communication technology for public safety.

KB: I think they should know that the results from the study are directly from first responders. It’s really important to have that perspective in any R&D, especially when there are so many problems that first responders currently face with their technology. We want to make sure those problems aren’t occurring in the future and that new problems aren’t unintentionally created for this group. It’s so important that first responders are involved and that their voices are highlighted. I believe that will be the best way to develop technology to have an impactful change on the technology first responders use to do their jobs safely, effectively, and efficiently.