Mixed Reality at work: The future of digital experience creation

What can Virtual Reality (VR) and Mixed Reality (MR) bring to the process of creating, curating, and composing digital experiences and content?

We're used to having a computer on our desk with a fixed, limited screen that displays the UI to create digital experiences as well as a preview of what we are building. But VR brings the opportunity to create outside the bounds of that rectangle. How could that help us?

At Magnolia, we like to explore and investigate emerging technology to understand the market and how to provide more value to our customers and partners. With the recent release of the Meta Quest Pro and the Apple Vision Pro, there is a lot of interest in "spatial computing".

And I see several clear intersections of VR and digital experience management.

Digital Experience Platforms (DXPs) enable non-technical users to create and manage websites and mobile apps. Similarly, they can also manage content for VR environments, including text, images, videos, and even 3D models and scenes.

Our customers are already using Magnolia’s content hub to deliver content to VR devices. To learn more, check out our blog Manage VR with a Headless CMS.

But what’s the best way to create 3D content and virtual experiences? In VR, of course.

Many VR apps include a building component in VR, trying to encourage their communities to create user-generated content, like Spatial, RecRoom, and Horizon Worlds. There are also VR apps specifically for 3D content, like MultiBrush, Gravity Sketch, Arkio, and Shapes XR.

We can leverage the new VR capabilities to improve the authoring experience of "traditional" non-VR content, too.

This is what I want to explore today.

The film "Minority Report" caught the audience’s attention with its film-noir data gloves. It envisioned a whole new way to interact with digital media.

Sci-fi films and television shows relish exploring novel interactions. It's fun to watch and helps us imagine how VR and hand-tracking could be used to create and work better. Though to be honest, many science fiction user interfaces (UIs) are just non-sensical eye candy.

But what if we could really leverage the power of VR and MR for our creative processes? What would that mean for a real-world use case? It means you can break free from the screen! With VR, the UI does not need to be bounded by a frame anymore. You are free to imagine any interaction. A driver's seat? The controls in a bus or a cargo truck? Construction equipment? An airplane cockpit?

For starters, VR headsets do provide web browsers, so it is theoretically already possible to work with a web-based content management system (CMS) using a VR headset. This might bring some value as you could have a very large monitor. But, after all, it is just a virtual monitor. How retro.

Can we use VR in a better, native way?

The heart of many DXPs and CMSs is the visual page builder that content authors use to create web experiences by adding components, content, and assets such as images. In VR, these elements would feel more like physical objects than on a 2D screen. I tried to imagine what this would feel like.

Naturally, as content authors, we want to conveniently drag and drop components onto a page. In VR, there's plenty of real estate, so we can have a palette of available components next to the page.

To drag a component, I can grab and move it like a real object. If I don't want a component any more, I can grab it and throw it away.

How would we place existing content on the page, like images from a DAM or products from an ecommerce system? Let's display these resources next to the page, too.

Having all content available at once is very comfortable. I can just reach over without changing the UI.

So, that's what I built. Have a look at this short video of a proof-of-concept that shows how Magnolia’s visual page editor could work in VR.

Above, we wondered what kind of cockpit we might want. Considering that most of us work on teams, how could VR benefit teams?

Insead of working in a small cockpit, we could work on the bridge of a ship with banks of controls and information for your team, including a communications panel, a navigation desk, and the conn.

A great thing about a visual page editor is an instant preview of the experience. And wouldn't it be nice to see how the experience would look on different devices?

With VR, we could pull up any device and actually (well, virtually) hold it and see the experience to ensure the responsive design looks great everywhere.

If you were working on digital signage, like in a cineplex, restaurant, or store (think H&M or Nike high-street stores) you could see the experience in the environment. You could also use this functionality to preview non-digital signage, for example, posters for seasonal campaigns.

I don't expect everyone to be wearing MR glasses in the next year. But I do anticipate more people using them as they get smaller and people discover how much they improve their work.

One use case I see today, is having a much bigger work canvas, allowing us to have access to our most relevant tools and content at the same time – just a glance to the left, right, up, or down away. Use case number two is having more extensive and realistic previews of content experiences.

And consider that you only need to bring your MR glasses with you; no laptop or monitors necessary. I predict this will be the killer app for VR and MR glasses.

I hope you enjoyed this thought experiment and would love to discuss it with you further on LinkedIn or Twitter.