Blog Series: Future of Smart Glasses #1

Augmented Reality (AR) has an interesting and almost supernatural appeal to it. This high-tech term is echoing through the corridors of most advanced industries today, bringing an image of a cyberpunk future or a high-tech cyborg to our minds. Possibilities of this fascinating technological frontier feel almost unlimited! Those could be our impressions with the most advanced of smart-glasses. Yet such impressions do not address the fundamental question: What is the purpose of Augmented Reality?

Once we are able to answer this question we will be equipped to do more objective analysis of the AR technology. Then we will understand the value of AR capable devices such as smart-glasses. In this blog series we will apply a divide and conquer approach in our journey on understanding the fundamentals of AR. We will set aside hype and our fascination for a while and look into the reality of augmented reality. Only then we can investigate the true value of AR and start selecting the best of technologies that serve our business needs.

We will take a look under the hood of AR and learn about the different technological aspects. This knowledge will then help us on deciding what type of technologies will enable us to create business value. Technologies that support the end-users in their mission to apply their human expertise for reaching their business goals. We will finally have our business augmented and reinforced with the best of technologies. After this journey we return back with more uniform picture of where this technological frontier is heading to.

The purpose of Augmented Reality is augmenting the human capabilities with the best of technologies in the context of our business operations. This augmentation will take us beyond our human limitations, enhancing us to go above what mankind has ever been capable of.

The ingredients of Augmented Reality

Augmented Reality is composed of four fundamental technology areas. Combinations of these areas are present in any AR capable device such as wearable smart-glasses or handheld mobile devices. These areas are combined to achieve the purpose of AR in augmenting human capabilities.

The four technological areas are:

• Display Devices (e.g. Smart-glasses, mobile phones…)
• Tracking Systems (e.g. LIDAR, image recognition…)
• Interaction Metaphors (e.g. Voice commands, gesture controls…)
• Augmented Solutions (e.g. Remote collaboration, 3D model viewers…)

These four areas are combined in different configurations to create an AR system.

The ingredients of AR are display devices, tracking systems, interaction metaphors and augmented solutions as we now know. This grouping is important for our next steps in learning about the differences and nuances between various type of commercially available AR systems. Most importantly this division guides us on how to select AR systems that serve our business needs. For this purpose we need to go through these technological realms individually and separately.

Introduction to Display Devices

The purpose of display devices is characterized by how we see our AR content. Display devices like diffractive waveguides, or more generally devices such as smart-glasses or mobile phones using various display technologies, are the first and most recognizable feature of Augmented Reality. This is the tangible thing that we encounter when we see examples of AR. Such examples could be an user observing the world through a mobile tablet display or a technician wearing smart-glasses when troubleshooting equipment failure. Behind the display technology group is a device that attempts to visualize information to user, in the case of smart-glasses a display device that is worn very close to users eyes with various optical characteristics allowing the display to superimpose users field of view with the content of our augmented solution.

When we look beyond the fascination we realize that we are just trying to find a optical display arrangement, handheld or wearable, that allows us to visualize information on top of our view to the real world. This we need to achieve while minimizing interference from our AR technology to the users other operations within the constraint of the business case. In respect to the smart-glasses, often requirement is simply having a hands-free display. All this in the context of a business case: technician needing both hands to service high complexity machinery while requiring additional guidance during the task execution.

Display devices have the ability to visualize information to the end user needs, hands-free for the smart-glasses or always in the pocket for standard mobile devices.

Introduction to Tracking Systems

The purpose of tracking systems is characterized by where we position our AR content.

Tracking systems like inertial measurement units, or more generally tracking frameworks such as ARCore or ARKit combining various tracking sensors, are the second and most intuitive feature of Augmented Reality. The most intuitive thing that we encounter when we experience examples of AR. For example an user observing the world with tablet device that is running our augmented solution. Content now stays aligned with the real world by the utilization of the tracking technology. Goal of the tracking frameworks is to extract spatial information of the world through a set of sensors. In the case of any of our AR capable devices a tracking system combines cameras, LIDAR, inertial measurement units and uses this tracking data for two purposes: 1. environmental understanding to create real-time 3D representation of the world around the user and 2. motion tracking to understand the display device and user position and orientation inside that 3D representation of the world.

When we look beyond the complexity we realize that we are just trying to find a sensor arrangement that allows us to extract information about the world around us. This we need to achieve while minimizing the interference to the users other operations within the constraint of the business case. In respect to the tracking technologies, often requirement is simply that of object recognition and display devices relative position and orientation in respect to the objects in the environment. All this in the context of a business case where technician needs to service high complexity machinery that require additional guidance during the task execution. Now enhanced with tracking that enables the AR content to be overlaid into a correct position and orientation in respect to the users field of view.

Tracking systems have the ability to spatially position information for the end user needs, providing information in right place based on the use-case context.

Introduction to Interaction Metaphors

The purpose of interaction metaphors is characterized by how we manipulate our AR content.

Interaction metaphors like gesture controls, or more generally user input systems such as touch screens or keyboard and mouse using various interaction technologies, are the third and most diverse feature of Augmented Reality. The interactive thing that we encounter when we experience examples of AR. For example a user observing the world with augmented solution, manipulating it with various interaction metaphors depending on the device they are using. Behind the interactivity ordeal is a system that attempts to capture intentions of the user, in the case of smart-glasses a voice recognition system that is capturing user speech. Users speech is then analyzed various audio processing algorithms allowing the user voice commands to manipulate the augmented solutions by speech instead or along with other means e.g. hand gestures.

When we look beyond the fascination we realize that we are just trying to find a input metaphor, active or passive, that allows the AR system to extract information about the user intentions. This needs to be achieved without interference to the users other operations within the constraint of the business case. In the case of interaction metaphors, often narrowed down to use case of clicking buttons or moving virtual items. All this in the context of a business case where technician needs to service high complexity machinery, while requiring additional guidance during the task execution. Now enhanced with interactions that allow this user to manipulate the AR system when needed.

Interaction metaphors have the ability to interpret intentions of the user, manipulate the information in a way that best serve the use-case context.

Introduction to Augmented Solutions

The purpose of augmented solutions is characterized by what we do with our AR content.

Augmented solutions like remote collaboration (e.g. POINTR), or more generally software solutions capable of augmented reality combining the first three capabilities, are the fourth and the value creating feature of Augmented Reality. The productivity increasing thing that we encounter when we experience examples of AR. For example a user interacting with the world together with augmented solution, solving real world problems faster and with less errors. This vision forms the expectation of augmented solution capabilities, when summoning thoughts about AR systems and their use in the real world. Behind the augmented solution is a system that attempts to enhance the capabilities of the user, viewed through smart-glasses, positioned with tracking system and manipulated through interaction metaphors. A service technician having back office knowledge and advice together with them allowing them to perform their tasks better than before.

Conclusions before the Deep Dive

We want to combine best of technologies, in the context of business case constraints, that maximize the capabilities of our augmented user to create value.

Conclusion what we can draw from all the four aspects is that smart-glasses and AR generally, can be divided into technical terms, our four fundamental technological groups. We are talking about display devices visualizing the content, e.g. how AR looks like to us, tracking systems positioning the content, e.g. where AR content is experienced, interaction metaphors for content manipulation, e.g. how we interact with AR, the content of our augmented software solution, e.g. what we experience as our AR content. The AR content, solution itself, implements the business case requirements for value creation. Solution that is operated on a smart-glass device when needed or handheld mobile device when deployed to existing mobile device base.

In the next episode of our blog series we will be going into the details of each individual technology group. With this first overview under our belt, we can start diving deeper into each of the four areas and see what is really important and what is just good-to-know. We will then be able to make knowledgeable decisions when adopting smart-glasses and standard mobile devices with AR capabilities for our business needs.

Meanwhile let us know if you have questions or if your would like us to help address your AR needs.

Contact us on how to get started with remote collaboration and smart-glasses!