It's all about timing. When the Apple Vision Pro first came out, it was ahead of its time—it could immerse users into its mixed-reality interface but its features were underused. Now that the software (VisionOS 2; more developer apps) has caught up, the Apple Vision Pro is finally doing what it's destined for. And with the release of Submerged, we finally see how the Apple Vision Pro is starting to cook.
Submerged is the first scripted short film filmed entirely in Apple Immersive Video. Directed by Oscar-winner Edward Berger (who adapted and directed All Quiet on the Western Front), the WWII thriller puts users in the middle of the action.
The plot follows a sinking submarine (upon re-reading this article, "sinking submarine" sounds... weird) and its crew, trying to survive. Tor Myhren, Apple’s VP of Marketing Communications, describes Submerged as the next evolution in storytelling. “Apple Vision Pro places you inside the story—inside a packed submarine alongside the crew. The depth of immersion here just wasn’t possible before. We’re thrilled to see how filmmakers like Edward Berger will push the boundaries with this technology.”
Filmed over three weeks, a full-scale, 23-tonne submarine set was constructed to resemble the WWII vessels and was designed for full submersion. Great lengths went into employing practical camera traps and special effects to create an immersive environment. The cast went through extensive stunt rehearsals to keep the action realistic.
Berger's approach to filmmaking was redefined by Apple’s immersive tech. “This new medium expands the horizon of storytelling. Apple Vision Pro allowed us to create a story in a way that was previously unimaginable. It’s a game-changer for filmmakers.”
Submerged is the first in many immersive content for Apple Vision Pro. Next month, Apple teams up with The Weeknd for an immersive music experience to celebrate the diamond-certified artist’s upcoming album, Hurry Up Tomorrow. Fans can also look forward to Concert for One, an intimate new concert series launching later this year, kicking off with a special set from award-winning British artist RAYE and new episodes of Adventure will be released later in the year.
And if you ask us, given the immersion of the Apple Vision Pro, maybe we can expect a horror short film/interactive game?
When it comes to prognostication, look to the dreamers, who imagine the things that will be made possible. For Apple, it's the sort of longview that benefits from the company's long development cycle for its products. It is this sort of gestation period that allows the Apple Vision Pro (AVP) to come to fruition.
In 2007, a patent was granted to Apple for a "HMD (head-mounted display)", which meant that development for the Apple Vision Pro ran for 16 years. That's the problem with dreamers, it takes a while before everybody, including the technology, catches up. But while most of the tech needed to be invented, some things like the look of the device didn't veer too far from a concept sketch.
"When we started this project, almost none of these tech existed," said Mike Rockwell, VP of Apple’s Vision Products Group. "We had to invent almost everything to make it happen."
Alan Dye, Apple’s Vice President of Human Interface Design, said that the AVP was, by far, the most ambitious Apple product that they had to design. "I can't believe that any other company would be able to make something like this as it requires all disciplines across the studio together, to create one singular product experience. It's kinda unprecedented."
Richard Howarth, Vice President of Industrial Design, concurs. "One of the reasons that it was very ambitious was that it hadn't been done. Nothing with this sort of resolution and computing power had ever been done.
"We didn't even know if it was possible."
The prototype was huge. Powered by a roomful of computers. Thick cables run between them. Although a behemoth, the prototype represented proof that it was possible.
Yes, during development, there were VR headsets that were released to the public. But Dye and Howarth weren't interested in creating a VR headset; they wanted a way to bridge people. Dye explains that whenever someone dons a VR headset, they are isolated from other people around them. "We wanted [the Apple Vision Pro] to foster connection, both by bringing people, from across the world, right into your space or by remaining connected with people around you."
The intent to connect framed how the product was designed. Like EyeSight, where your eyes—or a simulacrum of your eyes—appear on the front of the Apple Vision Pro if you're addressing someone or if you're using an app (an animation plays letting others know that you can't see them). Essentially, it's a visual aid for others to know whether you're available or not.
Mixing AR and VR, the Apple Vision Pro would pioneer "spatial computing", where the integration of digital information into the user's physical environment. The only way that could work was Apple's proprietary M2 chip that powers the device and an R1 spatial co-processor. Another way for the AVP to process the workload is "foveated rendering", where it only renders what your eyes are looking at.
The micro‑OLED display puts out 23 million pixels didn't hurt either. There are also 12 cameras for precise inside-out tracking (it tracks your eyes, your hand gestures and human bodies who come within your ambit).
Dye and Howarth didn't want to use external controllers and opted for hand gestures and voice commands to get around. The hardware is only as good as the software. That's where visionOS comes in.
visionOS lets you create your own Persona, an almost realistic avatar of yourself or allows for the aforementioned EyeSight. It still is kinda janky (eye tracking is left wanting if I'm selecting something at the edge of my periphery).
But still, the visionOS won the prestigious D&AD Black Pencil award for Digital Design and a Silver Cannes Lion for Digital Craft. The judges saw potential and there's still the visionOS 2 on the horizon, where the update allows for a functioning Magic Keyboard to appear in a virtual environment or customise the icons' position on the home screen.
One of the features that I look forward to is creating spatial photos from photos from the Photos app library. Using advanced machine learning, the visionOS turns a 2D image into a spatial photo that comes to life on the AVP.
It was only a few years ago, that the Google Glasses was slammed by the public for being too intrusive. While cultural norms have shifted to the point where the public is lax about their privacy, Apple is still up in arms about privacy and security.
"It's important to know that Vision Pro has a privacy-first design at its core. We took great care in privacy and security for it," Rockwell says. "We don't give camera access to the developers directly. When your eyes highlight a UI element, the developers won't know where your eye position is. They are only informed if you tap on something. Another thing is that if you're capturing a spatial video or spatial photo, it alerts others on the front display that you're recording."
The Apple Vision Pro retails for SGD5,299 and there will be many, who would baulk at that price.
"We built an incredible product that we believe has enormous value," Rockwell explains. "This is not a toy. It's a very powerful tool. It's intended to be something that can give you computing capabilities, the ability to use this in a way where there's nothing else out there that can do it. We reached into the future to pull back a bunch of technology to make it happen.
"We want to ensure that this has fundamental and intrinsic value and we believe that at the price, it is of good value."
Perhaps access to the future is worth the ticket? The Apple Vision Pro is emblematic of the promise of the imminent. Of the convenience of speaking with your loved ones or the experience to traipse in lands unseen.
Like the eyes of the oracle, the device brims with potential and given time, the future is more realised.
The Apple Vision Pro is out now.
Let's start with science fiction and how we imagine it—the time travelling; phasers; light sabers. It's what makes the future so alluring. That the things we imagine are made real. Of course, there are always the pesky constraints of real-world physics that prevent such wonders to stay shackled in the realm of the mind. But sometimes a little stubbornness goes a long way. Such is the case of Apple and its entry into the mixed reality game: the Vision Pro.
From your View-Masters (remember those) to the Oculus Rift, we have been creating "headsets that immerse you into another reality". (To set the record straight, we're not talking about augmented reality, which is digital content overlaid over the real world but mixed reality that integrates digital objects into the user's environment.)
Apple may not have pioneered mixed reality but it sure is gonna leave its competitor in its wake of "spatial computing".
We tried the Apple Vision Pro (or the AVP, which shares the same initialism with Aliens Versus Predator) and the visuals are, for the lack of a better word, magical. It's magical that you're able to look at an icon and double tapping your fingertips would open up the programme. It's magical that you don't get the bends from being in an immersive video. And, it is so magical that you can open up multiple windows and... work became fun? It felt like that Jonny Mnemonic scene.
One of the ways that the AVP is able to process the workload is a sneaky thing called "foveated rendering". Because it tracks your eye, it only renders what your eyes are looking at: stare at a window and it comes into clear. Look at another window and that becomes sharp. If you think about it, that's how our eyes work anyway.
The hardware of this is incredible. Made of magnesium and carbon fibre, there are twelve cameras—from tracking of your hands to spatial tracking—positioned throughout the headset. There's an M2 processor and an R1 spatial co-processor to deliver a smooth performance. The eye tracking is a cinch and there's no lag in the video passthrough.
On the corners of the goggles are a digital crown that adjusts the volume and the immersion and a button that you can depress to take photos and videos. There are speakers fixed to the arms of the Vision Pro but if the volume goes past a certain level, everybody else around you are privy to what you're hearing.
The AVP's Persona feature is kinda weird. Think of a Persona as your avatar. Your Pesona will reflect youryour facial expressions (sticking out your tongue; gesticulate with your hands), it has fringes of the Uncanny Valley. It. You can FaceTime or enter into an online meeting with them; they would appear and the hairs on your arm will rise a little. But after a while, you get used to it. And then their Personas kinda look like ghosts in your living room. Except they are presenting a PowerPoint.
If you're wondering, why not use a memoji? And the only reason I can think of is that if you're in a business meeting, there has to be a level of professionalism so a unicorn or a poop memoji may not fly. Then, again, it would be nice to have options. Perhaps in the next VisionOS upgrade.
By the way, there's an announcement that there would be a VisionOS 2, where you can create spatial photos from your 2D images, have new gesture controls and an enhanced Persona—accurate skin tone, clothing colour options. Who knows, maybe there would be an inclusion of memojis?
The price is expensive. Like SGD5,299 expensive. But that's to justify the years of R&D and the components. You hold the AVP in your hands and it feels nice. And I suspect that months later, people wouldn't blink at the price tag. I remember when mobile phones retailed at four digits and my uncle self thought, welp, I'm not paying that much for a compact supercomputer. A year or two later, that sort of pricing for a mobile phone became normalise.
To fit in all that goodness that makes the AVP work its magic, it will have some weight to it. To be fair, it weighs about 649g. That's equivalent to a medium-sized chinchilla or a bag of Cadbury Triple Pack Mixed Eggs. Not that heavy, right? But when you're wearing the AVP that's outfitted with a Solo Knit Band on your face, after a while, you're gonna feel it in your face and because of my terrible posture, my neck will compensate for the weight and I'll hunch even further.
As a remedy, you can swap out the Solo Knit Band for the Dual Loop Band, which gives better weight distribution. Or, if you're a stubborn cock like me and you find it leceh to change to a Dual Loop Band, you can wear it lying down.
If you're worried about the tension in your neck, don't worry; you'll know its time to put down the AVP when it runs out of battery at two hours of general use.
I kid.
It's not perfect but this is a game changer. It possesses the tech of today to The AVP shown what is possible and yet also poses what else can be done. We don't think that Apple is done with the Vision Pro; there's a roadmap and it's gonna take a few generations of the AVP before it gets to that stage, where you can't ignore it any longer. Like the first-gen iPod or the first-gen iPhone, the AVP has raised the bar and the other brands are gonna have to play catch-up.
It's a promise of a future, one that is bright with potential and all it took was an Apple Vision Pro for that glimpse.
The Apple Vision Pro is out now.
