Introduction

We started experimenting with Mixed Reality in 2013. Since this was before the arrival of Google Glass, Microsoft Hololens, and Magic Leap, we used available consumer devices to create an augmented glasses. These glasses provided an augmented experience using Google StreetView. These early experiments provided the fundamental insights for exaQuark, a Mixed Reality platform which can scale to millions of users.

Our augmented glasses

At the time of our earliest experiments, augmented glasses weren’t easily accessible. We built a custom set using available consumer devices:

- an Epson Moverio glasses. This is a wifi enabled device that runs Android
- a Samsung Galaxy S3 mini smartphone
- a clamp, strong tape, etc. to interlock the two android devices.

The Epson Moverio connects to the internet via the mobile phone. The app for our “hybrid glass” has 2 parts: one running on the glasses, another one on the smartphone. The smartphone performs most of the work, tracking the user’s location and the direction they are facing.

Creating an augmented reality

Google StreetView offers an amazingly complex “digital replica” of the world, which performs a lot of the heavy lifting for Mixed Reality. For our experiments we built HybridEarth, which uses StreetView and a custom-built digital representation of our office. HybridEarth contains digital avatars of “remote” users who can roam around the world.

It also contains “local” users (wearing AR glasses). Local users are mapped to the digital world, as shown on the screen in the first video:

From the AR glasses, “real users” can see the remote avatars mapped into their physical location. Here is an example of what they see:

Next steps

This year we are hoping to see AR devices become more mainstream. If there is interest in HybridEarth we will open it up to for users to explore and develop their own mixed reality environments, using StreetView as a common map to explore between the environments.