Over the past 10 years, many potential applications of augmented reality (AR) for urban infrastructure have been proposed: visualization of subsurface utilities, asset identification and query, excavation and construction planning, etc. Unfortunately, the implementation of AR apps that would be sufficiently accurate and robust for use in infrastructure engineering is hard – because accurately augmenting the physical world is challenging.
For a long time, the main difficulty that AR developers had to face is the real time estimation of the position of the tablet or phone doing the augmentation. Tracking is fundamental to AR, and inaccurate position estimation leads to inaccurate augmentation. Although recent augmentation devices such as the Google Tango or the Microsoft HoloLens pretty much solved the tracking problem in indoor environments, accurate outdoor augmentation is still challenging. Most tracking techniques rely on tracking visual features in a video stream captured by a phone or tablet. Vehicle movements, wind, changing shadows caused by clouds, and uniform surfaces all affect the number of reliable features to be tracked, which results in shaky augmentations, for which the accuracy can hardly be estimated, let alone be predicted. Infrastructure workers cannot rely on such low accuracy augmentation for their work.
Working on the outdoor augmentation problem, our team came up with a solution to the accuracy issue: instead of trying to track the physical world, we "capture" it. Watch our system in action:
Of course, our solution is not true AR. I guess we could call it mesh augmentation, or augmentation of mirror worlds. However in the context of urban infrastructure engineering, it has several advantages over true AR systems:
However, our technique also has disadvantages compared to true AR:
Some may question the viability of such a technique, considering the requirement of using an up to date high resolution reality mesh of an entire city. Such meshes, however, are becoming more common every year – take for instance this mesh of the city of Paris, that was obtained in a few months from aerial photos, for an area over 1000 km2.
I am convinced that in the future, most major cities will use such photorealistic meshes for urban management and planning.
The method we proposed is not as flexible as true AR. It is actually a tradeoff that has the advantage of offering accurate augmentations that users can rely on, that could be developed today using existing technology, and that would require, on site, nothing more than a tablet or a phone…