The NIST Smart Space Project

Pervasive devices, sensors, and networks, provide infrastructure for context-aware smart meeting rooms that sense ongoing human activities and respond to them. These technologies require advances in areas including networking, distributed computing, sensor data acquisition, signal processing, speech recognition, human identification, and natural language processing. Open interoperability and metrology standards for the sensor and recognition technologies can aid research and development programs in making these advances. To address this need the NIST Smart Space and Meeting Recognition projects are developing tools for data formats, transport, distributed processing, and metadata. We are using them to create annotated multimodal research corpora and measurement algorithms for smart meeting rooms, which we are making available to interested members of the research and development community.

The NIST mission is to address measurement, standards and interoperability challenges that must be met as tools for industrial Research and Development laboratories worldwide.

The Meeting Room Recognition Project application

Smart Spaces require large amounts of sensor data to be transported to networked processing resources. We have defined an abstract data transport mechanism and a reference implementation for the Linux Operating System to support research in the area. We hope you will join us, and our several existing contributors in further developing this system. It is fully in the public domain. Our primary internal application of this system is in the NIST Meeting Recognition project. The figures below show the meeting room in operation on its review station, and the NIST Smart Data Flow map that performs the streaming media transfer and distributed processing necessary for the data collection. We handle data streams from two hundred eighty microphones and seven HD video cameras, and produce data files that can be retrieved by time stamp. This data is used to support recognition processing research communities working in audio, video, and sensor fusions. These generate two-hundred gigabytes per hour of data on many nodes on a gigabit network. This technology will be important in the office of the future, and in command and situation management centers.

Figure 1. The NIST Meeting Room Data Collection Laboratory in operation. The review station shows seven views; six in thumb-nail, and one at center. Level meters for the twenty-four random placement microphones and four microphone arrays are at bottom.

Figure 2. the NIST Smart Data Flow System data flow map for the meeting room at the Smart Flow Control Center showing the twenty-seven data flow nodes, and the flow connections that represent the data transport.

Figure 3. The NIST Mark-III Microphone Array v2 was created to enable our research partners in developing techniques for speech recognition, speaker identification, source localization, and source separation in meetings.

Summary

The Smart Space Project works in the areas of:

• High data rate sensor data transport and processing
• Unique sensor technology needed for Smart Spaces
• Signal quality measurements for speech

The infrastructure and measurement tools that we make available to the research and development community are having positive impacts on efforts to develop next generation of multimodal computer interfaces. Please follow the link to see our open reference implementations of sensor data flow, microphone array, and new speech SNR metric, and the link to Some Partners and Users to see some of our partnerships. If you are interested in the project, please contact us to discuss collaborative research and development.