Early clinical tests show real-time picture-in-picture tech may restore navigational awareness in hemianopia
After a stroke knocked out part of his visual field, Glenn Calverley found himself walking into signposts, missing cats and second-guessing his every step. Now, an experimental mixed reality device from the University of Alberta is helping patients like him recover some of that lost visual awareness.1
In a recent proof-of-concept study published in the Journal of Neuro-Ophthalmology, a Canadian research team reported that their prototype software, built for Microsoft’s HoloLens 2, helped stroke survivors with homonymous hemianopia (HH) avoid collisions while walking a 50-meter obstacle course.2 The glasses create a real-time miniaturized view of the missing visual field and project it into the area patients can still see. The effect is something like a wearable digital periscope.
When half your world goes missing after a stroke
Homonymous hemianopia is a common but often invisible consequence of stroke and traumatic brain injury. It occurs when damage to the visual pathways in the brain causes the loss of one half of the visual field in both eyes. Unlike peripheral vision loss from ocular disease, HH originates entirely from the brain, leaving patients with functional eyes but a persistent blind side.
This isn’t just a clinical oddity. According to the American Stroke Association, up to 30 percent of stroke survivors experience some form of visual field loss.3 For those affected, everyday activities become minefields. People with HH frequently bump into unseen objects, struggle with reading, and are unable to drive or navigate unfamiliar environments safely. Existing tools such as prisms and scanning training provide limited benefit, not fully compensating for missing visual input.4
READ MORE: Neuralink’s Blindsight Implant Receives FDA Breakthrough Device Designation
For patients like Calverley, who found himself missing movement from the left side of his field, the result was a quiet erosion of freedom. “I don’t see out of the corner of my eye,” he explained. “I can be walking down the sidewalk and walk into a signpost because I don’t see it.”
Until now, no assistive technology had offered a reliable way to restore real-time visual awareness in HH patients. But that’s what Dr. Edsel Ing and his team set out to change.
Mixed reality meets clinical neurology
In response to the limited rehabilitation tools for patients with HH, Dr. Edsel Ing and colleagues at the University of Alberta developed a prototype navigation aid using Microsoft HoloLens 2. The system uses mixed reality technology to overlay visual information from the impaired visual field into the patient’s intact hemifield, improving spatial awareness and obstacle detection.1
It’s a digital detour around a neurological roadblock. Built on the Unity platform, the software enables real-time picture-in-picture navigation (PIPN). A head-mounted camera captures the user’s surroundings and transposes a miniature live feed into their remaining field of view. The system uses WebSockets for real-time communication and calibrates via a browser-based interface powered by Streamlit.
READ MORE: The Digital Revolution is not Coming to Ophthalmology. It has Arrived!
Unlike virtual reality headsets that block out the real world entirely, the HoloLens mixed reality design lets users see their environment while layering in compressed visual data. The prototype system increased functional perception by transposing 52 diagonal degrees of lost peripheral vision into the patient’s intact field.
According to Dr. Ing, the goal was to create something usable, accessible and shareable. The system isn’t locked behind a patent or paywall. Dr. Ing and his collaborators published the code as open source, hoping patients, clinicians and developers will build on it, refine it and eventually help bring it to market.
“These glasses give hemianopia patients the perception of the whole world back again,” said Dr. Ing.
Clinical testing shows functional gains in real-world settings
The research team conducted a crossover study to evaluate the prototype’s effectiveness in improving navigation among patients with HH.2 Five participants completed a 50-meter obstacle course both with and without the mixed reality glasses. Each course included four foam obstacles, designed to simulate common mobility hazards.
When not using the device, three out of five participants collided with all four obstacles. When wearing the mixed reality glasses, only one participant experienced a single collision. Walk times increased by an average of six seconds while using the glasses, but this difference did not reach statistical significance.
Patients were asked to rate the glasses’ impact on their ability to navigate. The median helpfulness rating was 74.3 out of 100, with participants citing improved spatial awareness and increased confidence while walking. On average, the glasses were perceived as 19.7% more helpful for ambulation, a statistically significant improvement (P = 0.028).2
These outcomes suggest that mixed reality PIPN can be a meaningful functional aid. Importantly, the device did not impede performance, and it offered measurable improvements in obstacle detection and subjective navigation confidence.
READ MORE: Helping Hands for the Visually Impaired
Adapting tech to meet real-world neurological needs
While the prototype remains in early development, the research team has taken steps to make it accessible. The software, which runs on the Microsoft HoloLens 2 platform (approximate retail cost: C$6,000), is available as open-source code. Dr. Ing and his colleagues hope that other clinicians, developers and even patients will adapt and refine it for wider use or eventual commercialization.
Beyond hemianopia, the team is beginning to explore how this technology might support patients with other neurological vision impairments, including torsional diplopia and nystagmus. These symptoms remain difficult to manage with conventional therapies and often lack effective assistive technologies.4
“There’s still no way to repair the damaged visual pathways in the brain,” said Dr. Ing in a university statement. “Until we get there, the responsibility falls to clinicians and technologists to come up with smarter, more responsive solutions.”
For now, this headset is not a cure, but it may be a step toward providing spatial awareness where none existed, and giving stroke and brain injury patients a clearer path through the world around them.
References
- Rutherford G. Mixed reality glasses give ‘the whole world back’ to people with partial vision loss. Folio (University of Alberta). 2025. Available at: https://www.ualberta.ca/en/folio/2025/05/mixed-reality-glasses-give-whole-world-back.html Accessed on May 27, 2025.
- Ing EB, Roy I, Tavakoli M, et al. Mixed reality glasses with picture-in-picture navigation for patients with homonymous hemianopic visual field loss. J Neuroophthalmol. 2025;45(2):215–218.
- American Heart Association. Visual disturbances. 2024. Available at: https://www.stroke.org/en/about-stroke/effects-of-stroke/vision-and-hearing/visual-disturbances Accessed on May 27, 2025.
- Persons J. National clinical trial aiming to improve vision rehabilitation. Massachusetts College of Pharmacy and Health Sciences. 2024. Available at: https://www.mcphs.edu/news/national-clinical-trial-aiming-to-improve-vision-rehabilitation Accessed on May 27, 2025.
