At Zappar we’ve been living and breathing immersive technologies for over 13 years bringing the most accessible tech solutions to the broadest range of people as possible. And so, as you may imagine, we’ve been itching to get our hands on Apple’s mixed reality headset, Vision Pro.
Apple provides several ways for building immersive experiences for Vision Pro, including Unity and Xcode, but the one we’re particularly interested in at Zappar is WebXR. Experiences built for WebXR are distributed over the web - accessed through the web browsers included with headsets - and that’s a game changer for a number of reasons:
In this blog, we’ll take a technical deep dive into Vision Pro’s support for WebXR. We’ll see that Apple Vision Pro is a great device for experiencing interactive WebXR content, but there are some important implementation details that make Vision Pro a bit different from other VR/MR headsets.
We’ll cover:
Let’s get started!
WebXR is a set of standards that allow web pages to offer immersive 3D content, including virtual reality, augmented reality, and mixed reality experiences. These standards are implemented (to varying degrees of completeness) by the web browsers included with VR/MR headsets, such as the Meta Quest 1/2/3/Pro, the Magic Leap 2 and (as we’ll see) Vision Pro.
The promise of WebXR is simple, build content once, targeting the WebXR specification, and have it run without modification on these different headsets. While that’s true in principle, in practice it’s somewhat complicated by the degree of implementation of the standards by web browsers, and the differences in form factor and input mechanisms (e.g. handheld controllers versus hand tracking) of the various headsets. As we’ll see throughout this deep dive, the Vision Pro is a relatively unique device, with corresponding opportunities and challenges for delivering XR content over the web.
In many cases it’s possible for developer tools to address these differences in implementation, allowing content creators to build experiences that run consistently across a wide range of devices. That’s exactly what Mattercraft does - it’s our fully featured development environment for building 3D web-based experiences, and its WebXR support provides a range of abstractions that make building rich and interactive content for headsets a breeze.
Apple considers the WebXR support on Vision Pro to be ‘experimental’ and requires the user to enable it in the Safari settings:
From the Settings app, head to Apps, then Safari, Advanced, and Feature Flags then enable the WebXR Device API and WebXR Hand Input Module. It’s then best to restart Safari to ensure that any pages are reloaded with the correct features enabled.
While this isn’t ideal for end users, it’s a process that only needs to be completed once. After it’s in place, a prompt to enter ‘Immersive Mode’ will be presented to the user whenever they visit a web page in Safari that requests a WebXR session.
The WebXR specification supports two primary modes of operation:
While Vision Pro is conceptually a ‘mixed reality’ device, showing the user their real-world environment, its WebXR implementation currently only supports the immersive-vr mode. This means that experiences must provide a background or virtual 3D environment.
In Mattercraft, we’ve added the ability to choose a primary WebXR mode and a ‘fallback’ mode for a given project. That means you can deploy experiences that run in mixed reality on devices that support it (such as Quest, Magic Leap, and our Zapbox headset) but still present a working (fully virtual) user experience on devices that don’t (like Vision Pro).
The WebXR specification provides several ways for users to interact with content, including:
To simplify the development of interactive WebXR experiences, Mattercraft provides a feature called ‘pointer emulation’. With it, developers build experiences using the point-and-click events that are common when building a traditional web page (e.g. ‘click’, ‘pointerdown’, ‘pointerup’). Mattercraft automatically maps the input mechanisms (either controllers or hands) exposed by any given headset to these events using rays in 3D space.
Vision Pro is not shipped with any controllers. Instead, users interact with its interface and apps using an ‘eye gaze plus finger/thumb pinch’ mechanism. While this feels intuitive and somewhat natural, Apple has unfortunately not exposed it through their WebXR implementation. We believe they’re working to influence the specification going forward to include abstractions for this kind of input, so we’re hopeful that this may implemented in the future.
There’s one other important feature of hand-tracking implementations to note - the two ‘anchors’ typically associated with a given hand. These are points in space that the device tracks the position of and makes available to WebXR content. They are:
Of this pair, Vision Pro implements only target ray space. This is not wholly unexpected - grip space is optional in the WebXR specification - so, to make things easier for content developers, Mattercraft emulates grip space using a point in space offset from the wrist location reported by Safari.
Once you’ve built and deployed a WebXR experience, the next step is to get users trying it out! Since WebXR experiences are web pages, distributing your content is as simple as distributing a web link. Vision Pro has a number of mechanisms that may be of use in this regard:
It’s hugely important, when building for any platform, that the process for developers is comfortable, productive and efficient. A great developer experience is one that lets the developer spend the most time thinking about what they’re trying to achieve and less about how they’re going to achieve it. In this regard, Vision Pro has some great features that set it apart from other headsets.
The most prominent of these features is ‘Mac Virtual Display’. While wearing a Vision Pro, looking at a ‘real life’ Mac in your immediate area shows you a ‘Connect’ button. Selecting this opens a large virtual screen for that Mac in front of you that you can use to operate without having to remove the headset. With this feature, you can build WebXR content on the virtual Mac using your existing development workflow and tooling, while seamlessly testing and previewing that content in the Vision Pro browser. It’s perhaps easier to show than describe, so we’ve recorded a short video demonstrating it.
Needless to say, we’ve made sure that Mattercraft’s Live Preview feature works great with this workflow. It makes for an almost instantaneous iteration time between change and testing when developing an XR experience.
Also of note with regards to developer experience is that the Vision Pro is relatively comfortable to wear for moderate amounts of time, particularly if using the ‘dual loop’ alternative headstrap included in the box. While the ‘Virtual Display’ doesn’t quite reach the quality of a real-life monitor, text (and code) is still sufficiently crisp to be read and written with ease.
As we can see, even with the ‘experimental’ nature of the headset’s WebXR support, it’s possible to build high-quality, interactive XR experiences for Vision Pro for deployment over the web. The quality of the headset itself, along with some great features for developers, mean it’s a pleasure to build content with and for.
That said, it’s clear that other headsets, such as Quest and Zapbox, have fantastic overall value propositions, including price point, user input options (aka controllers), and maturity/breadth of support for the WebXR specification. The beauty of the cross-platform nature of WebXR, in tandem with great tooling, is that you don’t have to put all your device eggs in one basket!
If you have any questions or comments regarding WebXR, Vision Pro, or Mattercraft then we’d love to hear from you! Feel free to drop into our Discord server, or drop our award-winning customer success team a message.
Start building immersive web experiences for Vision Pro today with a free 14-day trial of Mattercraft.