How the HoloLens could facilitate issue investigation and 3D pipe design in industrial facilities

Facility maintenance work may be repetitive: sets of instructions need to be executed on a regular basis, to ensure continued operation of the facility.  As explained in my previous post, an Augmented Reality (AR) tutor could help new employees familiarize themselves and complete the task, by teaching them how to proceed, on a step by step basis.  Experienced employees could also benefit from the tutor, using it as a check list.  However, one potential danger is facility employees could end up following instructions blindly, ignoring their former training.  While AR maintenance tutors would do an excellent job in most cases, they would dramatically fail in unexpected situations, as they could give users instructions that are inappropriate for the situation.

Virtual tutors make sense when the steps necessary to execute the task are known in advance.  But what if something unexpected happens?  For instance, a pipe bursts, or an instrument stops working properly. A tutor app would not be of any use here, because it has not been programmed to deal with such situations.  In such cases the user is on his own…

Unfortunately, such critical decisions are often constrained by time and by the availability of information.  Instrument reading history, history of past events, notes left by colleagues, drawings and device specifications could all be useful to the user when trying to decide on how to solve the problem. But the time that would be required to collect and analyze all that information might make the situation even more critical. 

As a solution to this, we proposed the concept of an AR assistant.  It would take the form of an Augmented Reality voice operated system that the user could ask questions to, and that would provide him with all the data he needs to resolve the issue he is facing. 

Such an assistant would not only save the user a lot of time, but it would enable him to base his decision on relevant contextual data that is delivered to him where and when he needs it. Our demo highlights several innovative services offered by the assistant:

  • A connection between a P&ID drawing and the physical world, saving the user from having to find the object that corresponds to a specific P&ID element, and facilitating his understanding of the drawing, as shown in a previous post;
  • A method of viewing and browsing pre-recorded data on a holographic 2D graph;
  • A way of designing pipe layouts on site, enabling the user to readily see and avoid clashes with existing pipes;
  • A method of viewing the hologram of a colleague at exactly the same physical location as during recording – this will be the subject of a future post.

Such an AR assistant application would of course depend on the existence and online availability of context data. The Internet of Things would offer a part of the solution, by providing online access to all connected instruments in a plant. 

One can easily envision the facility of the future where employees could inspect, operate, maintain and repair assets highly efficiently, by having timely access to all sorts of contextual data related with those assets.  Augmented Reality would give them some sort of superpowers, enabling them to achieve more, and more accurately, in less time.  Such future is closer than we think...