Author: Louis Rosenberg, PHD
🔎 The Inside Scoop: Why We Handpicked This Article for Repost
You might be familiar with the implementation of augmented reality in gaming, e-commerce, but have you ever thought it could offer more than just pleasure? It is likely that AR could empower us with abilities that were once deemed as science fiction in healthcare and daily-life scenarios. Let’s explore the boundless possibilities augmented reality could bring us. (By VIVERSE Editorial Team)
Over the next decade your mobile phone will be replaced by AR glasses that you will wear during most of your waking hours. The first wave of products will hit the market by 2025. While many consumers, myself included, are skeptical that we’d ever want to wear hardware on our faces for hours each day… we will.
The reason: Augmented Reality will Give Us Superpowers.
I know you’re unconvinced. Having been involved in AR and VR from the very beginning, and I fully appreciate the cynicism. The industry lived through a colossal false start in the 1990s, and another in the 2010s.
But this time is different. Immersive technologies are finally able to offer real value to real users (and I’m not talking about gamers). While I can point to many examples, no field makes the case more clearly than medicine.
Leading the way: Medical Superpowers
Earlier this month I participated in the Digital Orthopedics Conference in San Francisco, where a major theme was to imagine medicine in the 2030s. As part of this effort, a small group of us carefully reviewed the latest research and assessed the potential impact of immersive technologies.
I have to admit, I was deeply impressed by how far augmented reality has progressed recently for use in medicine. I expect that by 2030, AR headsets will be a common tool for surgeons, radiologists, and many other medical professionals. That’s because AR will give doctors superhuman abilities to visualize medical images, patient data, and other clinical content
The first Superpower: X-Ray Vision
Augmented Reality will give doctors X-ray vision — the ability to peer into a patient and see trauma or disease at the exact location in their body where it resides. Of course, doctors can already look under the skin using CT scans and MRIs, but these images are displayed on flat screens, forcing professionals to imagine how the medical content relates to the patient on the table. This is an impressive skill, but it takes time and mental effort.
With AR headsets, the superpower of x-ray vision is now a reality. In an impressive study from Teikyo University’s School of Medicine, researchers tested an experimental emergency room with the ability to capture whole-body CT scans of trauma patients. The medical team wore AR headsets, allowing personnel to peer into the patient on the operating table and see the trauma. This allowed the team to discuss the injuries and plan the treatment without referring to a flat screen. It saved time, reduced distractions, and cut down on the need to extrapolate from limited information.
Another superpower: Enhanced Dexterity
When performing delicate procedures, AR will provide surgeons with navigational cues projected onto and into the patient in real time. This guidance will increase accuracy, reduce mental effort, and shorten the time taken. The potential value for surgery is extreme, from minimally invasive procedures such as laparoscopy and endoscopy, to freehand surgical efforts such as placing orthopedic implants.
In fact, I predict augmented reality will become required equipment for many surgical procedures within the next ten years. I say that as someone who has been a champion of augmented surgery for decades, as the idea goes back to the first AR system — the Virtual Fixtures platform — developed at the Air Force Research Laboratory in the early 1990s. The goal of that early effort was to show that AR could boost dexterity in precision tasks such as surgery.
To appreciate the progress the field has made over the last 30 years, consider this: when testing AR for dexterity enhancement in the early 1990s, the Virtual Fixtures system required users to move metal pegs between holes spaced two feet apart. Now, thirty years later, surgeons at Johns Hopkins, Thomas Jefferson University Hospitals, and Washington University performed a delicate spinal procedure on 28 patients. They used AR to help place metal screws with a precision of 2 mm. As published in a recent study, the system registered the virtual overlays to the real patient with such accuracy that surgeons scored 98% on standard performance metrics.
Looking forward, we can expect augmented reality to impact all aspects of medicine. That is because its precision has reached clinically viable levels. In addition, major breakthroughs are in the works that will make it faster and easier to use augmented reality in medical settings. The biggest challenge for medical AR is to accurately align virtual content with the real patient. This currently means attaching physical markers to the body, which takes time and effort. In a recent study from Imperial College London and the University of Pisa, researchers tested a marker-less AR system that uses cameras and AI to accurately align the AR content. The system worked, suggesting that time consuming medical markers will soon be a thing of the past.
The superpowers of X-ray vision and dexterity enhancement will certainly go beyond medicine to other fields from construction and engineering, to manufacturing and agriculture. But the biggest impact will be on mainstream consumers, who will have a wide range of superpowers for daily use.
Mainstream Superpowers: Omniscience and Super-Recall
With augmented reality, mainstream consumers will soon be able to gaze almost anything in their surroundings, squint thoughtfully, and instantly view informational content. From encyclopedic information about any person, place or thing in your field of view, to price-checks of any product you see in a store, in a street, or even in a friends' house — instant information is coming. A quick head nod to confirm you’ve seen it, and the information will fade away. These capabilities are coming and they will feel like magic.
Augmented reality will also give you superhuman recall. It will be impossible to forget someone’s name, as this will appear the instant you see them, along with the names of their spouse and kids. You’ll even be reminded of the last time you met and what you discussed, all of this seamlessly provided.
A household superpower: Telekinesis
Augmented Reality will also integrate with smart homes, allowing users to glance at a light switch across the room and give a quick gesture (like a character in Harry Potter) and it will magically turn on. This will feel like a true superpower, and it will extend beyond lights to all electronic devices. With glances and gestures we will learn to control our augmented world.
And when it comes to music, the eyewear will allow you to do more than just listen — it will have the ability to alter the look of your surroundings in sync with the song, making it feel like you are suddenly transported into your own real-time music video as you walk down the street.
This brings me back to my thesis: Over the next ten years, augmented reality will replace the mobile phone as our primary interface for digital content. Early adopters will embrace the lure of new, magical capabilities. Everyone else, skeptics included, will quickly find themselves at a disadvantage without omniscience, x-ray vision, superhuman recall, and dozens of other capabilities that are not even on the drawing board yet. This will drive adoption as quickly as the transition from flip phones to smartphones. After all, not upgrading your hardware will mean missing out on layers of useful information that everyone else can see.
An augmented world is coming. It will be magically embellished with artistic content and infused with superhuman abilities. But there are also risks we must avoid, as augmented reality will give tech platforms unprecedented ability to track our activities and mediate our experiences. For these reasons, we need to push for a safe and regulated metaverse, especially the augmented metaverse. It will get here sooner than you think.
This article originally appeared in BIG THINK (May 2022).
Papers Mentioned
Hu X, Baena FRY, Cutolo F. Head-Mounted Augmented Reality Platform for Markerless Orthopaedic Navigation. IEEE J Biomed Health Inform. 2022 Feb;26(2):910–921. doi: 10.1109/JBHI.2021.3088442. Epub 2022 Feb 4. PMID: 34115600.
Rosenberg, L.B. (2022). Augmented Reality: Reflections at Thirty Years. In: Arai, K. (eds) Proceedings of the Future Technologies Conference (FTC) 2021, Volume 1. FTC 2021. Lecture Notes in Networks and Systems, vol 358. Springer, Cham. https://doi.org/10.1007/978-3-030-89906-6_1
Liu, A., Jin, Y., Cottrill, E., Khan, M., Westbroek, E., Ehresman, J., Pennington, Z., Lo, S. L., Sciubba, D. M., Molina, C. A., & Witham, T. F. (2022). Clinical accuracy and initial experience with augmented reality–assisted pedicle screw placement: the first 205 screws, Journal of Neurosurgery: Spine, 36(3), 351–357. Retrieved May 29, 2022, from https://thejns.org/spine/view/journals/j-neurosurg-spine/36/3/article-p351.xml
Ito K, Sugimoto M, Tsunoyama T, Nagao T, Kondo H, Nakazawa K, Tomonaga A, Miyake Y, Sakamoto T. A trauma patient care simulation using extended reality technology in the hybrid emergency room system. J Trauma Acute Care Surg. 2021 May 1;90(5):e108-e112. doi: 10.1097/TA.0000000000003086. PMID: 33797500.
L. B. Rosenberg. The Use of Virtual Fixtures As Perceptual Overlays to Enhance Operator Performance in Remote Environments. Technical Report AL-TR-0089, USAF Armstrong Laboratory, Wright-Patterson AFB OH, 1992. https://apps.dtic.mil/sti/citations/ADA292450
This post was originally published on Augmented Reality will give us Superpowers.
✍️ About Louis Rosenberg, PHD
AI Researcher. VR & AR pioneer. Founder of Unanimous AI. Founder of Immersion Corp. Founder of Outland Research. PhD Stanford. Over 300 patents for VR, AR, AI.
Learn more: https://medium.com/@louis_rosenberg