A Robust Method for Resolving Incorrect Visual Occlusion in Dynamic Augmented Reality Environments of Animated Engineering Operations: The objective of this project is to investigate and design a functional and accurate occlusion handling method that can be easily integrated into any Augmented Reality (AR) application. AR is an emerging visualization technology which blends computer generated scenes in the form of CAD objects with the views of the real environment. From the point of view of an observer of an AR scene, incorrect visual occlusion occurs when part or whole of a real object must block the observer’s view of a virtual (i.e. CAD) object. If not resolved accurately, incorrect occlusion leads to unconvincing visual artifacts that reduce the observer’s confidence and reliability in the AR visual simulations, and consequently prevents them and other decision makers from comfortably using the simulation results for making crucial decisions.

 

This research proposes a novel integration of geometry capture, remote sensing, and data communication technologies to enable realistic, reliable, and visually convincing results in AR animations of engineering operations. The research objective is being achieved by developing an automated method to obtain, record, and retrieve the depth information for both virtual and real entities in an AR animated scene of an engineering operation using remote sensing devices, and a z-buffering technique to detect and resolve any identified incorrect visual occlusion instances in real-time. The results are applicable in solving a wide range of engineering problems in construction, civil engineering, manufacturing, mechanical design, shipbuilding, transportation, and other domains. Please visit the publications page to download technical papers describing different aspects of this research in more detail. Sample animations of conducted experiments can also be viewed on the videos page.

Cyber-enabled Wireless Monitoring Systems for the Protection of Deteriorating National Infrastructure Systems: The long-term deterioration of large-scale infrastructure systems is a complex national problem that, if left unchecked, could lead to catastrophes on the same scale as the I-35W Bridge collapse (August 2007). The goal of this NIST TIP project is to demonstrate a comprehensive structural monitoring system assembled from transformative sensor technologies to prevent future catastrophes from occurring within the aging fleet of infrastructure systems. Specifically, a cyber-enabled monitoring system is being designed to focus upon the identification of fatigue and corrosion in bridges, two closely related degradation mechanisms that if left undetected lead to brittle structural failures offering little, to no warning. To fully address the complex task of identifying such serious damage conditions, the monitoring system is being designed to span across multiple length-scales beginning at the sub-structural component and spanning up to the regional scale consisting of bridge fleets.

 

The Laboratory for Interactive Visualization in Engineering (LIVE) is leading the research thrust on the design and implementation of context-aware information visualization solutions. The objective of this task is to investigate methods to facilitate efficient interaction between human inspectors in the field and the pervasive sensor network that will monitor the state of a bridge or supporting structures. The specific goal of the research is to design and implement a context-aware mobile computing technique that will be capable of automatically identifying an inspector’s spatial context, and retrieve bridge information and sensor data that is of relevance to the decision contemplated at a particular time and location. Please visit the publications page to download technical papers describing different aspects of this research in more detail. Sample animations of conducted experiments can also be viewed on the videos page.

Interactive 3D Visualization of Simulated Construction Operations in Outdoor Augmented Reality:  The objective of this research is to develop an extensible and scalable Augmented Reality (AR) platform to facilitate the dynamic 3D visualization of simulated construction and other engineering operations in an outdoor augmented environment. Related issues being addressed during the course of this research include tracking and registration of CAD objects in a global coordinate system, manipulation of an arbitrary number of virtual (i.e. CAD) objects performing an operation over a real background, establishing modular communication methods for data acquisition from the geo-positioning and orientation tracking devices, integration with project schedules and electronic terrain maps, design and implementation of a general-purpose animation authoring interface, and the design and implementation of an ergonomic mobile AR backpack.

The first tangible result of this ongoing project is the ARVISCOPE visualization system. ARVISCOPE is an AR based visualization tool driven by a powerful animation authoring language that can create dynamic animated scenes of simulated operations in construction and other engineering domains. 3D animations created in ARVISCOPE are accurate and faithful graphical representations of underlying Discrete-Event Simulation (DES) models. ARVISCOPE is a fully mobile visualization platform that takes advantage of state-of-the-art position and orientation tracking technologies to update the contents of an user's view of the augmented space continuously based on the current line of sight. Please visit the publications page to download technical papers describing different aspects of this research in more detail. Sample animations created in ARVISCOPE can also be viewed on the videos page.

Rapid Post-Disaster Reconnaissance for Building Damage using Augmented Situational Visualization and Simulation Technology: The objective of this research is to develop and implement a novel reconnaissance methodology for rapid and accurate quantification of structural damage sustained by buildings during natural or human perpetrated disasters. The technology being developed consists of an Augmented Reality (AR) see-through display, a high-accuracy GPS receiver, and a magnetic orientation tracker all connected to a light-weight computing platform. Using information from the attached sensors, the system acquires visual field data via an attached camera and georeferences it with computer simulated images in real time. This can allow on-site users to superimpose previously stored building information onto the corresponding view of a real structure in an AR setting, and estimate damage by evaluating the discrepancy between the two views. Field users can then translate the measured discrepancies into damage indices that can allow critical decisions about a building’s structural integrity and safety to be rapidly made.

The technology being developed will deliver unprecedented on-site capabilities to building inspectors and first responders evaluating building damage in the aftermath of catastrophic events such as terrorist attacks and earthquakes in urban areas. In addition, by using feedback information from the actual view of a building, users will be able to update structural analysis models and conduct on-site what-if simulations to explore how a building might collapse if critical structural members fail, or how the building’s stability could best be enhanced by strengthening key structural members. This project is being jointly pursued with my colleague Prof. Sherif El-Tawil. Please visit the publications page to download technical papers describing different aspects of this research in more detail.

Location-Aware Contextual Information Access and Retrieval for Rapid On-Site Decision Making in Construction, Inspection, Maintenance, and Urban Disasters: The objective of this research is to investigate the requirements, design, implement, validate, and demonstrate a new, self-contained, location-aware Information Technology (IT). This technology will be capable of automatically providing engineers, inspectors, and first responders with accurate, prioritized contextual information for making critical, real-time decisions on construction and maintenance sites, and in chaotic, post-disaster environments. The proposed research is critical because the inability to identify and access relevant information is the primary obstacle that prevents rapid and optimal decision-making by constructors and inspectors, as well as by emergency responders (e.g., firefighters, civil engineers) who respond to natural and manmade disasters.

The research objective is being achieved by developing a mobile user context-sensing framework that accurately tracks an engineer’s or first responder’s three-dimensional spatial context in any indoor and/or outdoor environment without relying on pre-installed sensors or trackers. Specific information of interest at a given time is then being retrieved by interpreting spatial context with a level of precision sufficiently high to accurately prioritize identified contextual data. Please visit the publications page to download technical papers describing different aspects of this research in more detail.

Inverse Kinematics and Interoperability Standards for Visualization of Construction Activities at the Operations Level of Detail: This research investigated methods to enable the 3D animation of a construction operation to be automatically generated from the kinematic properties of the resources (e.g. equipment, craftsmen) that perform that operation, and the geometry of the infrastructure (e.g. building). This has been achieved by designing an analysis technique that considers an articulated resource to be a system of linkages that can be analyzed as an open kinematic chain. This allows the implement of such a resource (e.g. excavator’s bucket, mason’s arm, etc.) to be considered as the kinematic chain’s end-effector that can be manipulated and controlled using goal-oriented techniques based on inverse kinematics. The goals of the end-effectors themselves, i.e. positions where a resource’s implement should be at key instances during an operation can be automatically extracted from interoperable 3D product models of facilities (e.g. in CIS/2 format).

A simple, software-authorable, object-oriented language to define articulated resources has been designed. External software processes (e.g. a Discrete-Event Simulation model) can instantiate resources defined in this language inside 3D virtual worlds, and instruct them to perform operations using a high-level, construction work-like terminology. This allows the automated 3D animation of construction operations of any length and complexity. Performance of actual field operations can be improved by allowing proper communication of the planned work prior to its execution. In addition to communicating what may happen in the future (by visualizing simulated operations), it is possible to automatically re-create in virtual worlds what happened in the past, and what is currently happening (from real-time data). Please visit the publications page to download technical papers describing different aspects of this research in more detail.