Longview: How the Apple Vision Pro Came to Be

From a germ of an idea to the vision of the future, here's how the much-anticipated Apple Vision Pro came to be
Published: 18 July 2024
(DAN WINTERS)

When it comes to prognostication, look to the dreamers, who imagine the things that will be made possible. For Apple, it's the sort of longview that benefits from the company's long development cycle for its products. It is this sort of gestation period that allows the Apple Vision Pro (AVP) to come to fruition.

In 2007, a patent was granted to Apple for a "HMD (head-mounted display)", which meant that development for the Apple Vision Pro ran for 16 years. That's the problem with dreamers, it takes a while before everybody, including the technology, catches up. But while most of the tech needed to be invented, some things like the look of the device didn't veer too far from a concept sketch.

"When we started this project, almost none of these tech existed," said Mike Rockwell, VP of Apple’s Vision Products Group. "We had to invent almost everything to make it happen."

Alan Dye, Apple’s Vice President of Human Interface Design, said that the AVP was, by far, the most ambitious Apple product that they had to design. "I can't believe that any other company would be able to make something like this as it requires all disciplines across the studio together, to create one singular product experience. It's kinda unprecedented."

Richard Howarth, Vice President of Industrial Design, concurs. "One of the reasons that it was very ambitious was that it hadn't been done. Nothing with this sort of resolution and computing power had ever been done.

"We didn't even know if it was possible."


The prototype was huge. Powered by a roomful of computers. Thick cables run between them. Although a behemoth, the prototype represented proof that it was possible.

Yes, during development, there were VR headsets that were released to the public. But Dye and Howarth weren't interested in creating a VR headset; they wanted a way to bridge people. Dye explains that whenever someone dons a VR headset, they are isolated from other people around them. "We wanted [the Apple Vision Pro] to foster connection, both by bringing people, from across the world, right into your space or by remaining connected with people around you."

The intent to connect framed how the product was designed. Like EyeSight, where your eyes—or a simulacrum of your eyes—appear on the front of the Apple Vision Pro if you're addressing someone or if you're using an app (an animation plays letting others know that you can't see them). Essentially, it's a visual aid for others to know whether you're available or not.

Mixing AR and VR, the Apple Vision Pro would pioneer "spatial computing", where the integration of digital information into the user's physical environment. The only way that could work was Apple's proprietary M2 chip that powers the device and an R1 spatial co-processor. Another way for the AVP to process the workload is "foveated rendering", where it only renders what your eyes are looking at.

The micro‑OLED display puts out 23 million pixels didn't hurt either. There are also 12 cameras for precise inside-out tracking (it tracks your eyes, your hand gestures and human bodies who come within your ambit).

Dye and Howarth didn't want to use external controllers and opted for hand gestures and voice commands to get around. The hardware is only as good as the software. That's where visionOS comes in.

visionOS lets you create your own Persona, an almost realistic avatar of yourself or allows for the aforementioned EyeSight. It still is kinda janky (eye tracking is left wanting if I'm selecting something at the edge of my periphery).

But still, the visionOS won the prestigious D&AD Black Pencil award for Digital Design and a Silver Cannes Lion for Digital Craft. The judges saw potential and there's still the visionOS 2 on the horizon, where the update allows for a functioning Magic Keyboard to appear in a virtual environment or customise the icons' position on the home screen.

One of the features that I look forward to is creating spatial photos from photos from the Photos app library. Using advanced machine learning, the visionOS turns a 2D image into a spatial photo that comes to life on the AVP.


(DAN WINTERS)

It was only a few years ago, that the Google Glasses was slammed by the public for being too intrusive. While cultural norms have shifted to the point where the public is lax about their privacy, Apple is still up in arms about privacy and security.

"It's important to know that Vision Pro has a privacy-first design at its core. We took great care in privacy and security for it," Rockwell says. "We don't give camera access to the developers directly. When your eyes highlight a UI element, the developers won't know where your eye position is. They are only informed if you tap on something. Another thing is that if you're capturing a spatial video or spatial photo, it alerts others on the front display that you're recording."

The Apple Vision Pro retails for SGD5,299 and there will be many, who would baulk at that price.

"We built an incredible product that we believe has enormous value," Rockwell explains. "This is not a toy. It's a very powerful tool. It's intended to be something that can give you computing capabilities, the ability to use this in a way where there's nothing else out there that can do it. We reached into the future to pull back a bunch of technology to make it happen.

"We want to ensure that this has fundamental and intrinsic value and we believe that at the price, it is of good value."

Perhaps access to the future is worth the ticket? The Apple Vision Pro is emblematic of the promise of the imminent. Of the convenience of speaking with your loved ones or the experience to traipse in lands unseen.

Like the eyes of the oracle, the device brims with potential and given time, the future is more realised.

The Apple Vision Pro is out now.

related posts

crosschevron-down