From multiplayer AR games to business meetings with holographic participants, collaborative AR allows users to interact with virtual content together.
It can feel weird to watch a person who’s enjoying AR content on their phone. They’re pointing the phone at something interesting in front of them, smiling, taking pictures of it… yet you see nothing. And even if you point your own phone at the exact same spot, you’ll still see nothing. Only if the person lets you look at their screen will you discover what that fascinating AR thing was – perhaps even a lion filling half of the room!
But what if there were a way for both of you to explore the same AR experience simultaneously, each from your own device? To perceive a 3D model as located in the same spot, but from different angles, as you would any real object?
Turns out that it’s possible. It’s called collaborative AR, or multiplayer AR. And you can use it for many more things than just games.
The value of collaborative AR
Defining multi-user AR experiences is easy. It’s just several people experiencing the same AR content at the same time – but from different devices. The users can be located in the same physical space or many miles away from each other. You could also say that they share the virtual world, though not necessarily the real world.
But why is collaborative AR such a big deal? Well, because there are two key applications of augmented reality where you really need this multiplayer element.
1) Business. The most interesting application of augmented reality in business is virtual meetings – but don’t confuse those with VR meetings! In VR, you meet your colleagues or business partners in a specially created virtual space, complete with 3D-designed walls, chairs, and potted plants. In an AR-enabled meeting, the other participants are projected into your own environment. So, when you and your teammate are in the office and John is projected from across the globe as a holograph, you should all be able to see him and hear him talk.
2) Gaming and entertainment. When you play a game with someone, you must share some elements of it – a gameboard, a ball, a piece of paper, and so forth. For example, if you’re playing beer pong with a friend, the positioning of the beer cups and balls has to be the same for both of you!
By the way, where do you think it’s more difficult to implement multi-user experiences – in AR or VR? In fact, collaborative virtual reality is just peanuts compared to multiplayer augmented reality. That’s because in AR, you have to track each user’s real space to position the 3D models correctly. For instance, occlusion, lighting, and shadows will be different for each player.
How Google does it: let iOS and Android users play together
Google’s ARCore has a nifty service called Cloud Anchors, which allows AR developers to create shared experiences for users on different platforms. Cloud anchors are just like normal anchors but – you guessed it – stored on a cloud! Here’s how they work:
1) You create an anchor locally on your device. You’ll need a recognizable point in space – say, the leg of a chair, rather than a blank spot on the floor. You’ll also need to move your phone around to capture the point well.
2) The anchor is uploaded to the ARCore Cloud Anchor Service.
3) The cloud service creates a map of the area and assigns a unique identifier to the point.
4) When another user points their phone at the same area, they can download the anchor data from the cloud to see the virtual object you placed there. The cloud service will determine the user’s position relative to the anchor and display the 3D model from the correct angle, with occlusion if needed, etc.
5) For now, you can include up to 20 cloud anchors, and they are stored on the server for 24 hours.
How Apple does it: building a common map
Apple’s multiplayer solution doesn’t require cloud storage of anchors. Each device creates its own anchors and shares it with nearby devices. Once they find a point in common, they can merge scenes and build a single AR map of the space. When the position of an anchor changes, it’s updated on all devices. The technology combines ARkit and RealityKit.
3 great collaborative AR cases
Multiuser augmented reality is still quite new, but these three projects demonstrate what it can become in the near future.
1) Spatial
Spatial is a holographic collaborative platform that can create 3D avatars for people from a regular photo. The avatars are very realistic – even their lips move in a lifelike way. You can use whiteboards and other virtual tools to present information, charts, etc. – and even stream the meeting for those not taking part.
For now, Spatial is available on just nine selected devices. AR meetings require a lot of bandwidth to run, so the startup has also partnered up with a few 5G carriers in Germany, Japan, and South Korea. By the way, the reliance on 5G is typical for next-generation AR and especially WebAR, where you don’t have to download any apps. We at ARzilla think that WebAR is the future, so we support the spread of fast 5G internet, too, though we know that some people are uneasy about it.
2) SwiftShot
This awesome AR game was presented at Apple’s WWDC20 conference. The goal is to knock over your opponent’s wooden blocks with a slingshot – though, of course, both the slingshots and the blocks are just 3D models. The occlusion is perfect, and the movement is so smooth that you almost forget the game props aren’t real.
Image: cnet.com
3) Niantic’s Neon
Niantic, the company behind Pokemon Go, is working on a massive set of tools for AR developers, called Planet-Scale AR Platform, and Neon is a demonstration of what it can do. Players have to collect white energy pellets from the ground and then use them to fight enemy targets:
Image: Niantic
Multiplayer apps will surely be in the AR spotlight in the next few years. However, you don’t need advanced collaborative games to attract attention to your brand – sometimes a simple SocialAR filter or mask can do wonders for engagement and sales. Drop us a line at elay@arzilla.io to tell us what your business is about, and we’ll happily share our insights!
