Product Digital Information Visualization and Exploration

Summary

The Product Digital Information Visualization and Exploration (ProDIVE) project will deliver standards, methods, and tools to improve interoperability of (i) data representations across the product lifecycle and (ii) product data standards with advanced visualization modalities, such as augmented reality (AR).  Past and on-going work at NIST have made significant gains in harmonizing product lifecycle data standards to help realize the so-called digital thread, a concept purported to construct persistent links across lifecycle activities, such as engineering design, manufacturing planning, fabrication, and inspection. To support additional use cases, this project will focus on the upstream propagation of downstream observations, such as inspection results, towards design processes, such as part requirements.

Registering manufacturing, sustainment, and other downstream data to a product-centric viewpoint will facilitate the wider adoption of industrial AR.  Primary barriers for industrial AR relate to the lack of interoperability across AR presentation and authoring systems with traditional product engineering software platforms.  This lack of interoperability presents challenges for (i) AR presentation systems to properly react to changes in upstream reference data, (ii) replicating authored AR scenes in similar manufacturing scenarios, such as similar but different assembly plans, and (iii) other scenarios related to spatial computing, such as linking observational data to spatial definitions.   

The ProDIVE project will build on earlier results, developing additional methods, protocols, and tools to (a) facilitate the projection of downstream product lifecycle data to upstream decision making, (b) establish a capability testing protocol for indoor localization technologies in manufacturing environments, and (c) improve interoperability of industrial augmented reality workflows.

Description

Objective - To deliver standards, methods, and tools that improve the usability and maintainability of product lifecycle data for (i) engineering design-focused analytics and (ii) advanced visualization modalities that leverage spatial computing technologies.  

What is the new technical idea? 

The manufacturing sector generates over 2 Exabytes of data a year, a figure expected to continue to grow (McKinsey Report 2010). To deal with the velocity, volume, and variety of these data, standards organizations and consortia have published frameworks to describe isolated stages of the product lifecycle, such as ISA-95 for manufacturing services. These efforts have been effective in addressing problems in some of those stages through activities such as automating manufacturing execution and managing material flow during production. However, these same efforts (i) still do not fully address the upstream flow of information for product design decision making and (ii) fail to adapt to disruptive technologies, such as industrial augmented reality (AR). 

The ProDIVE project will focus on a product-centric incorporation of spatial computing into engineering workflows. Spatial computing can be defined as "human interaction with a machine [that] … retains and manipulates referents to real objects and spaces” (Greenwold 2003).  Spatial computing is fundamental to enable AR experiences. The recent uptick in industrial AR installations centered around task-oriented maintenance, complex human-driven assembly, and other use cases, requires documentation and testing of best practices to ensure such implementations’ maintainability, updatability, and overall sustainment.  AR-enabling software and hardware are evolving rapidly to improve (i) mapping of physical spaces, (ii) detection and alignment of physical objects to virtual representations, and (iii) multi-user AR experiences for industrial applications. The diversity of these solutions leads to significant challenges in interoperability between them.  Relating product definition standards with well-conceived systems and analysis models will help alleviate interoperability challenges.

Increased harmonization across product lifecycle data representations will help enable an AR-capable digital thread. As a result, the ProDIVE project will take advantage of on-going work in projecting downstream product lifecycle data to upstream activities.  Related contributions include but are not limited to (i) mapping inspection results to design requirements through knowledge graphs, (ii) segmenting process plans by its execution data in the absence of persistent identifiers, and (iii) labelling large design model repositories to enable data-driven design.

What is the research plan?

Leveraging a product-centric perspective, the ProDIVE project will develop, test, and facilitate the adoption of standards and best practices that help manufacturers realize a more accessible, robust, agile, and AR-capable digital thread. 

Activities for developing standards and best practices will include bridging gaps within and across current computable representations of (a) product and manufacturing information (PMI) and (b) indoor spatial representations for manufacturing facilities (through location-based services).

Activities for testing conformance and applicability of standards and best practices will include implementing interactive prototypes to test conformance to standard geospatial application programming interfaces, representations, and frameworks with smart manufacturing concepts.

Activities for facilitating the adoption, implementation, and proper use of standards and best practices will include research and standardization roadmap activities that identify and prioritize gaps in critical standards.  Specific application areas include (a) product definition harmonization and (b) integration of product and manufacturing definitions with data representations salient for spatial computing.  To support these activities, we will continue development of translation mechanisms between three-dimensional (3D) model representations generated through engineering design activities for deployment on various visualization modalities.