Iron Man and Pokemon Go hold clues to the next leap in the Internet

Augmented reality, or AR, has legs, unlike the hollow-eyed avatars in the current cartoon-like worlds of the metaverse

I pinched my index finger and thumb, rotated my hand and stretched my arms apart in the air to manipulate the 3D hologram of a liver projected on a special pair of tinted glasses. So, this is what surgery is like.

My glimpse into the world of surgeons came via augmented reality (AR) lenses. The rather heavy see-through glasses were mounted on an adjustable headband on my head. Not quite Marvel superhero Tony Stark's omnipresent artificial-intelligence (AI) assistant Jarvis, which projects 3D holograms. But this pair of Microsoft HoloLens 2 I tried on two weeks ago is starting to transform the way people work.

National University Hospital surgeons in Singapore have used the 5G-connected HoloLens 2 to assist with more than 50 heart, brain, spinal cord and liver surgical procedures to date – said to be a first in South-east Asia. By August, the hospital plans to double the number of HoloLens 2 deployed to 50 sets, and avail their use to all surgeons. National University Health System assistant chief technology officer Gao Yujia said the glasses allow surgeons to better visualise patients' organs, tissues and surgical complications with the help of computer-generated information. Before, surgeons had to scrub out and exit the operating theatre to refer to patients' magnetic resonance imaging or computerised tomography scans.

"But now I can manipulate the 3D images while wearing my sterile gloves; I don't have to interrupt the surgery," said Dr Gao, who gave a demonstration of the technology at Microsoft Singapore's experience centre on May 11.

Could this be the new frontier of the Internet, with AR stealing a march on the VR (virtual reality) world of what people loosely call the "metaverse"?

The metaverse has several variants. According to Meta, it is an immersive 3D virtual community accessible by headsets for people represented by their avatars to hang out and collaborate. In its cartoon-like Meta Horizon Worlds, avatars are floating torsos with no legs. Popular online video game platforms Fortnite and Roblox also rely on the use of VR headsets to enter online worlds, while browser-based gaming platforms Decentraland and The Sandbox promote the sale of virtual real estate and transactions in cryptocurrencies.

Over the past two years, this form of the metaverse saw luxe brands, hotels and entertainment giants forking out millions of dollars for virtual land. For instance, Ralph Lauren set up a store in Roblox during Christmas to allow people to buy digital clothing for their avatars. Adidas, Atari, Ubisoft, Binance, Warner Music and Gucci also bought land in The Sandbox.

In 2022, Singaporean tycoon Kwek Leng Beng's Millennium Hotels and Resorts opened a new virtual M Social hotel in a prime spot in Decentraland. Also in 2022, DBS inked a deal to acquire a plot of virtual land in The Sandbox to host activities.

The noise has since died down. Investments in the metaverse have fallen on harder times as tech-dom refocuses on profits amid high interest rates and tighter cash flow. United States retail chain Walmart reportedly shut down its Universe of Play experience for children on Roblox just six months after its launch. In March, entertainment media giant Walt Disney Company closed its metaverse division.

Despite the fading enthusiasm, several home-grown firms – OCBC, CapitaLand and Changi Airport Group – in April launched virtual locations and hosted events on Decentraland and Roblox. The firms said they need to acquire new technologies know-how, even if it is just an experiment.

Mr Chris Chong, CapitaLand Investment's chief executive of retail and workspace in Singapore and Malaysia, told The Straits Times: "This whole discussion about whether to get on the metaverse currently is similar to discussions about whether you need to get on social media a few years ago... Today, you can't live without social media."

A NEW DEFINITION FOR THE METAVERSE

But people hanging out in purely digital spaces is not the only form of the metaverse. There is also the world of AR.

Apple is a major proponent. Microsoft also sees a future in AR; in its concept of an "industrial metaverse", visual cues overlaid on physical objects seen through AR glasses could help workers on a factory floor repair machines or provide safety training to industrial site workers. German engineering and technology firm Bosch uses the HoloLens 2 to train engineers on complex vehicle repairs.

Whatever the Internet evolves into next, I'm convinced it is unlikely to take the form of cartoon worlds and avatars offered by gaming and social media platforms. It is too tedious to manipulate an avatar with a computer mouse or a VR headset to get things done online.

Mr Terry Ray, chief technology officer of US-based data protection firm Imperva, likens dialling into these cartoon worlds to making voice calls with every social contact regularly just to stay connected. "I don't really want to pick up the phone and call them all the time. What's so great about Facebook and Instagram is that we can stay connected without having to put a lot of work in maintaining those connections," said Mr Ray.

Also, VR headsets cause cyber sickness often accompanied by nausea, dizziness and headaches. The Meta Oculus Quest 2, the most affordable piece of decent VR hardware available today at US$400 (S$541), made me feel sick just one minute into using it to navigate a virtual space. Its clumsy hand controls, needed to move my avatar's head and body, are also not the most intuitive.

If people are going to spend more time online, they are going to gravitate towards technologies that are comfortable and intuitive, and require low maintenance.

AR HAS LEGS

I believe the answer lies in AR. It is often described as a key part of the new iteration of the Internet. The good news? The concept is not new to the general masses.

AR's first big debut was in 2016 with the global launch of US-based game developer Niantic's Pokemon Go app. The game overlays a digital world of creatures and virtual areas of interest onto the real world. Players "catch" these cuddly creatures at various physical locations using their smartphone's cameras, sensors and geolocation features.

The game gained millions of users around the world within weeks of its launch. Some enthusiastic players even made headlines with questionable decisions to play at sensitive locations, including a military base in Indonesia, forcing authorities around the world to issue warnings against trespassing.

The Pokemon Go craze has since died down, but its impact continues to linger. In 2021, the National Gallery in London took the collections of the National Gallery, National Portrait Gallery and Royal Academy of Arts "outdoors" through AR.

The National Museum of Singapore has an immersive exhibition called Story of the Forest featuring 69 images from the William Farquhar Collection of Natural History Drawings. Through a smartphone app, museum visitors can hunt for and "catch" 3D plants and animals within the paintings.

Separately, South-east Asia's biggest Audi showroom in Singapore sports an AR app to allow visitors to "drive" – using a tablet in the showroom – the Audi R18 race car around a virtual track at full speed without the risk of getting a ticket. And even though Walmart ditched virtual gaming on Roblox, it recently acquired Israeli-founded AR optical tech company Memomi to build new e-commerce experiences such as virtual try-ons.

That said, swiping screens and tapping on virtual keyboards and apps belong in today's Internet age. The future Internet will likely centre around instinctual hand gestures and voice commands a la depictions in futuristic sci-fi and superhero movies. Wearable technology such as smart glasses – with built-in AI natural language chatbots, cameras that read hand gestures, and smart lenses that display text and images – will be key. Imagine speaking to smart glasses to schedule meetings without having to type furiously on virtual keyboards. The glasses could also work with a cloud-based language engine to automatically display translated subtitles on smart lenses when someone speaks in a foreign language. Another plus is that AR is less likely to induce motion sickness. I did not feel giddy using HoloLens 2. The lenses are transparent and allowed me to see my physical world. Dr Rolf Illenberger, founder and managing director of Germany-based VRdirect, which builds VR environments for businesses, said: "AR wearables will have the power to replace smartphones eventually as the key facilitator for digital services."

HURDLES TO OVERCOME

But in order to go mainstream, AR wearables will have to overcome some hurdles, including privacy concerns and price. People freaked out when Google Glass – equipped with a camera that users could activate any time without notifying those around them – was released in 2013. Another barrier is price. The HoloLens 2 has a starting price of US$3,500. These headsets are also bulky. Kudos to Meta for making the most stylish-looking Stories smart glasses in partnership with Ray-Ban. They are also affordable at US$299, which is a good entry point for consumers, said Associate Professor Damien Joseph, Nanyang Business School's associate dean of undergraduate academics.

But the Meta Stories glasses are a tough sell on functions, as they are mainly for shooting photos and videos for uploading onto Facebook. Thankfully, competition is heating up. New hardware players looking to enter the AR space could break down existing barriers and open up new possibilities. At the Mobile World Congress trade show in Barcelona in February, Chinese firm Xiaomi took the wraps off its prototype Xiaomi Wireless AR Glass Discovery Edition. But the glasses were also bulky.

All eyes are now on Apple, rumoured to be working on an AR headset to be announced in 2023 in its push to sell more products beyond smartphones. Apple hasn't acknowledged the rumours. But Bloomberg reported in April that the firm had plans to pack in all sorts of features including games, fitness services and an app for reading books in virtual reality. This approach would be similar to what it did in 2014 when it unveiled its Apple Watch, pitching it initially as a timepiece, fitness tracker, widgets viewer, walkie-talkie and Apple TV remote control before health tracking and notification became the clear winners. Imperva's Mr Ray is hoping for a breakthrough in design. "I'm not going to wear some big goofy things into a meeting. Maybe here's where Apple comes in to work with Porsche Design to make them look nice."

Whatever the new Internet brings, it needs to make the way we socialise, work and transact more efficient. This will be similar to how past generations of the Internet – from the static World Wide Web pages to the user-generated Web of social media – brought about social and economic benefits within a quarter of a century. It will be a good start if the upcoming new hardware could look like the AR sunglasses that superhero Stark sports in the Iron Man movies – even if the glasses don't give users access to Stark Industries' global satellite network and an arsenal of missiles and drones.

Whatever the Internet evolves into next, I'm convinced it is unlikely to take the form of cartoon worlds and avatars offered by gaming and social media platforms. It is too tedious to manipulate an avatar with a computer mouse or a VR headset to get things done online.

The future Internet will likely centre around instinctual hand gestures and voice commands a la depictions in futuristic sci-fi and superhero movies. Wearable technology such as smart glasses – with built-in AI natural language chatbots, cameras that read hand gestures, and smart lenses that display text and images – will be key.

 

Source : The Straits Times