AR in health care

The applications of AR in healthcare

In this series of articles we look at Augmented Reality (AR) (or Added Reality; TR). Earlier, we discussed what AR is and how it works (using sensor technology). We also discussed the applications of AR. In this article we look in more detail at one of the fields in which AR is used: care. AR in healthcare has the potential to improve medical training, save lives and make existing processes more accurate and efficient. Doctors can carry out operations more effectively. AR can also be used to improve fitness. AR technology can reshape the patient experience, improve clinical outcomes and deliver innovative new therapies. For example, AR apps are already being used for simulations, operations and rehabilitation.

Training

AR technology makes medical training interactive and immediate, so that students can link theory more quickly to reality and the associated consequences of actions. AR can facilitate medical training in areas such as anatomy and procedures. Students get a better idea of the human body through AR. For example, an application can place video, audio or 3D models over a skeleton. The student can ‘peel’ skin and muscles from the body, practice the names of bones and at the same time learn more about how the body works.

Through AR it is also possible to experience surgery through the eyes of a surgeon. This can be done on a tablet or smartphone. This is much more cost-effective than medical simulators, while at the same time reaching a much larger audience. Simulations can also create opportunities for (deliberate) failure, which is not typically possible in health care.

Because AR can be applied indefinitely at the time that suits the student best, this is highly appreciated by students (3). Because less material is used than in the regular training courses and students and teachers have to travel less, this also contributes a little to sustainability (1).

Study

AR can add multimedia information to literature. AR transforms the 2D image of the human heart into a 3D beating heart, with more detail than is possible in a diagram or photo. Books enriched with AR can be used in the same way for new therapies and medicines. In this way AR clarifies the images and text in the textbooks. The technology can also be used to streamline communication between professors and students, by using AR during class and practicing to show models.

Treatment

There are AR applications that locate the veins more accurately, help surgeons demonstrate their surgical plans and help better diagnose patients by placing CT scans and other images over a patient’s body. In addition, there are AR applications that can reconstruct tumors or other masses to be removed.

When it comes to detecting certain malignant conditions such as skin cancer, AR can monitor the problems early. High risk individuals can have their skin checked regularly and the AR app can quickly help the doctor compare results and changes over time. Another possibility is the real-time creation of digital models, for fillings and braces, among other things. AR can also be used for the treatment of phobias or the representation of the fetus in the uterus.

Operations

To bring AR into operating rooms, surgeons focus their glasses on a trigger on their patient – usually a temporary tattoo. They then see the procedure illustrated step by step with CT and MRI images, notes and diagnostic information lying on top of the patient’s skin. That way they know where the mass to be operated is and where they have to cut to create less trauma. Thanks to the technology, they can also see what it is doing in the patient.

The risks associated with (minimally invasive) surgeries can be reduced by presenting the most important information to the surgeon’s eye. Where surgeries previously required monitors with vital statistics and images taken by cameras, AR can ensure that a physician remains focused on the task during the procedure. Finally, with the help of AR, surgeons can also have their mentor present during the operation. For example, it allows a real-time projection of the mentor’s hands into the surgeon’s field of vision.

Patient interviews

Doctors can use AR to project clinical data into their field of vision when talking to patients. They see a summary of important information, such as diabetes, pacemaker, recent knee surgery. Changes in blood pressure and weight are marked. For example, this form of AR is triggered by a trigger outside the patient’s room, triggering information ranging from nurses’ notes to laboratory results.

Aftercare

After leaving the hospital, it may be necessary for a patient to continue certain treatments. For older people in particular, it can be quite challenging to keep track of what medication to take and when. AR apps can act as a digital health care provider in reminding people of medication intake or by encouraging them to eat better and exercise more. This can be combined with effects of not taking the medication or taking an overdose, or for example dietary suggestions based on the state of health.

There are AR apps that educate patients and are used, for example, to illustrate the impact of diseases. These apps can also help inform family members and carers about the pathology and consequences of certain diseases, including eye diseases (e.g. cataracts), which can be simulated using AR. AR also allows patients to scan packaging or printed materials to discover the benefits and efficacy of the drug. Patients gain direct access to the drug description simply by scanning.

Well-being

AR can be used to improve well-being. In this way, a person can be projected data about his workout and condition. The world around us can become our doctor and coach and help us make decisions: do you undergo that operation, take the stairs, leave that wine? AR can, among other things, show you the consequences and advantages of the options. AR apps can also provide reminders and guidance for everyday tasks such as shopping and travel. There are also AR games that get people moving or encourage them to remain physically and mentally active. Immersive video content can also be a welcome addition, enabling people to stay in touch with family and friends.

Resources

There are AR applications that help people with autism to learn social skills, and there are AR applications that allow healthcare providers to interact with breastfeeding mothers. We are also working on lenses with AR that can enrich reality, for example by turning the page of a book by blinking the eye. AR can also improve the vision itself. With digital contact lenses, retinal implants and portable technology it is possible to improve vision for people who are partially blind. These are designed to improve the visibility of objects or people. The AR application can detect walls, tables, doorways and more.