As soon as the patient is under anesthesia, the surgeons get to work. Precision is the order of the day, because the shorter the operation, the easier it is on the patient. In orthopedics, surgeons make sure this is the case by planning procedures in advance on the computer—especially when it comes to selecting prosthetics. For this they use software provided by a company called mediCAD Hectec. But how helpful would it be if surgeons could take a 3D version of their plans with them into the operating room? The answer: very helpful indeed. That’s why the company developed its mediCAD MR solution, which allows surgeons to use a HoloLens 2 headset so they can access and interact with the necessary information without contaminating the sterile field.

The challenge: Taking the plan into the OR

Working in a hospital is stressful. For many orthopedic surgeons, the day often starts as early as 8 a.m. with the first operation. Between surgeries, they have to fit in office hours and rounds and prepare for upcoming procedures. They keep up this hectic pace all day, but they never forget that each patient and each operation is unique. Proper planning is therefore essential. “We develop all our solutions in collaboration with experts in orthopedic surgery,” says Eric Erdmann, Head of Development at mediCAD Hectec. “Our goal is to create solutions that can be integrated seamlessly into day-to-day hospital life with a view to providing the best possible support for planning and performing operations.”

The company already offers mediCAD 3D, a tool for producing three-dimensional plans for orthopedic procedures during pre-op. This allows doctors to align implants and screws with greater precision than when using 2D planning tools. But as soon as they enter the OR, accessing the plan becomes rather tricky. “Some surgeons actually print out the plans and hang them up in the operating room. But the human body is often a good deal more complex than can be depicted on an A3 sheet of paper,” says David Würdinger, Product

Manager at mediCAD Hectec. “Of course, the OR is equipped with screens displaying information relating to the operation. The problem with this setup is that it’s time-consuming: the surgeon must step away from the patient, remove their gloves, operate the computer, look at the plans without the patient in front of them, put on new sterile gloves, return to the patient, and pick up again where they left off. That’s all rather inconvenient.”

mediCAD Hectec therefore decided to develop a better solution. What began as a research project back in 2018 is now reality and goes by the name mediCAD MR.

The solution: HoloLens 2 brings operation plans into the OR MR stands for mixed reality. This technology requires the right hardware, such as HoloLens 2, to project objects as holograms into the user’s field of vision. Objects like the plans for an operation, for instance. “After an operation has been planned in mediCAD 3D, you simply press a button and the data is prepared for use with HoloLens 2,” Würdinger says. “Thanks to our assistance app for mediCAD MR on the computer, users also have the option of enriching the data, for example with X-ray images or with the instructions for a new kind of implant, that they can refer to during the operation.” Once the data has been prepared, the software generates a QR code; when the surgeon views this code with HoloLens 2, the plans open in their field of vision.

Erdmann has talked to doctors who are already using the solution: “Everyone who’s worn HoloLens 2 in the OR is convinced that it’s a real game-changer. They have the plan images right in front of their eyes at actual size and in 3D, just like the patient, and can zoom in on tissue and bone structures, examine each individual layer, and rotate the view around any axis they wish.” And since the device is operated using voice commands and gestures, all this happens without contaminating the sterile field. “We design our solutions so that they dovetail seamlessly with hospital workflows. And we’ve done that yet again with mediCAD MR and HoloLens 2,” Würdinger says.

Another new feature that is available since recently is remote access. This allows those who have been granted permission to watch a first-person view of the images that HoloLens 2

livestreams from the OR. It’s also possible to communicate with the surgeon. “This opens up a whole new set of possibilities,” Würdinger says. One is that other specialists are able to advise during complicated operations. Another concerns representatives from the implant manufacturer, who are often present when surgeons are putting in a new kind of implant. “As things stand, the representatives have to go all the way to the hospital just to spend five or ten minutes in the OR. Remote access simplifies this workflow immensely,” Würdinger says.

The solution also harbors great potential for training aspiring surgeons. It’s possible for mediCAD MR to record operations, which is particularly useful for non-routine and thus rarely performed procedures. That way, students can discuss the operation step by step with their instructors afterwards. At present, hospitals using the solution do so only on an on-premises basis. But the goal is to tap the full potential of the Azure cloud platform. “Obviously, data protection, security, and reliability are top priorities for hospitals,” Erdmann says. “Microsoft fulfills all our requirements here in the cloud as well, for instance through services such as Azure confidential computing for data protection. As we set about developing the solution further, this gives us a massive cloud toolbox that we will definitely dip into in the future.”

“Everyone who’s worn HoloLens 2 in the OR is convinced that it’s a real game-changer. They have the plan images right in front of their eyes at actual size and in 3D, just like the patient.”

Eric Erdmann, Head of Development, mediCAD Hectec

“We design our solutions so that they dovetail seamlessly with hospital workflows. And we’ve done that yet again with mediCAD MR and HoloLens 2.”

David Würdinger, Product Manager, mediCAD Hectec