An official website of the United States government
Here’s how you know
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Incident Streams 2021 off the Deep End: Deeper Annotations and Evaluations in Twitter
Published
Author(s)
Cody Buntain, Richard McCreadie, Ian Soboroff
Abstract
This paper summarizes the final year of the four-year Incident Streams track (TREC-IS), which has produced a large dataset comprising 136,263 annotated tweets, spanning 98 crisis events. Goals of this final year were twofold: 1) to add new categories for assessing messages, with a focus on characterizing the audience, author, and images associated with these messages, and 2) to significantly enlarge the TREC-IS dataset with new events, with an emphasis of deeper pools for sampling. Beyond these two goals, TREC-IS has nearly doubled the number of annotated messages per event for the 26 crises introduced in 2021 and has released a new parallel dataset of 312,546 images associated with crisis content – with 7,297 tweets having annotations about their embedded images. Our analyses of this new crisis data yields new insights about the context of a tweet; e.g., messages intended for a local audience and those that contain images of weather forecasts and infographics have higher than average assessments of priority but are relatively rare. Tweets containing images, however, have significantly higher perceived priorities than tweets without images. Moving to deeper pools, while tending to lower classification performance, also does not generally impact performance rankings or alter distributions of information-types. We end this paper with a discussion of these datasets, analyses, their implications, and how they contribute both new data and insights to the broader crisis informatics community.
Proceedings Title
Proceedings of the 19th Information Systems for Crisis Response and Management Conference
Conference Dates
May 22-25, 2022
Conference Location
Tarbes, FR
Conference Title
Information Systems for Crisis Response and Management
Buntain, C.
, McCreadie, R.
and Soboroff, I.
(2022),
Incident Streams 2021 off the Deep End: Deeper Annotations and Evaluations in Twitter, Proceedings of the 19th Information Systems for Crisis Response and Management Conference, Tarbes, FR, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=934607
(Accessed August 14, 2024)