Patient-specific preoperative planning and use of mixed reality in shoulder arthroplasty in 2020


The shoulder is one of the most complex joints in the human body being an intricate combination of four different joints participating simultaneously in the motion of the shoulder girdle. Our understanding of shoulder pathology has increased significantly in the last decade and as a result a multitude of solutions are now available for joint preservation and joint replacement. The number of shoulder arthroplasties has increased constantly in the last 10 years, at a rate even higher than the lower extremity joint replacement procedures. Joint disease in the shoulder often leads to complex deformities; these need to be understood and addressed making shoulder arthroplasty an intricate balance between skill and technological prowess. Rapid and parallel technological developments have allowed for enhanced preoperative planning solutions and innovative tools for surgical execution. This article aims to present current patient-specific preoperative planning solutions and recent use of mixed reality technology in shoulder surgery.


  • Shoulder replacement procedures have shown a rapid growth and increased popularity over the last 10 years.

  • Preoperative planning of shoulder arthroplasty is
    crucial to ensure successful postoperative outcomes but challenges remain to surgically reproduce pre-planned implant positioning.

  • Mixed reality technology
    provides a promising solution to bridge the gap between pre-planning and intraoperative surgical execution.

The Australian National Joint Replacement Registry reported a 168% increase in shoulder replacement procedures since 2008. The Registry noted revision rates of 12% and 7% at 10 years for Total Shoulder Arthroplasty (TSA) and Reverse Shoulder Arthroplasty (RSA) respectively.1

Shoulder arthroplasty aims to restore glenohumeral range of motion and function and, to reduce pain. Accurate implant positioning is crucial to ensure a successful outcome of shoulder arthroplasty. Malpositioning of the glenoid component may lead to scapular notching, implant loosening or instability. In addition, glenoid bone defects are often encountered in shoulder arthroplasty as a result of cuff tear arthropathy with glenoid erosion, previous failed shoulder arthroplasty or chronic glenohumeral dislocation.5 Limited intraoperative view of the whole scapula; combined with difficult surgical exposure poses significant challenges for the orthopaedic surgeon intraoperatively. Preoperative evaluation of scapular and glenoid anatomy and surgical planning are crucial steps to ensure successful postoperative outcomes.

Patient-specific preoperative planning

Glenoid component malpositioning may lead to abnormal loading of the glenoid and accounts for up to 30–50% of complications following shoulder arthroplasty.10 Technological improvements allowed orthopaedic surgeons to shift from the traditional 2-Dimensional (2D) planning on plain radiographs and Computed Tomography (CT) scans to 3-Dimensional (3D) planning using specialised software. However, execution of the preoperative plan to reproduce the exact same intraoperative implant position remains elusive. Computer-Assisted Orthopaedic Surgery (CAOS) and/or use of Patient-Specific Instrumentation (PSI) were developed to help bridge the gap between pre-planning and intraoperative surgical execution.

Use of intraoperative computer navigation (also called CAOS) assists in implant positioning. A clinical study from 2009 investigated the accuracy of glenoid positioning in TSA. The authors demonstrated an improved accuracy in glenoid positioning compared to the freehand technique. However, the operating time was significantly longer in the CAOS group.6

PSI is another alternative to improve implant positioning intraoperatively. The preoperative plan created by the surgeon using dedicated proprietary planning software is sent to the medical device company for 3D printing of personalised PSI. The customised guidewire positioning allows the surgeons to reproduce the glenoid position as planned preoperatively. A review study from 2018 compared the ability of PSI and standard methods to reproduce the preoperative surgical planning of glenoid component positioning.10 From 12 studies involving 227 participants, deviations from the preoperative plan for version, inclination and entry point were significantly lower when PSI guides were used compared to freehand traditional techniques using standard instrumentation. In addition, the number of outliers were significantly higher with standard instrumentation than with use of PSI.10

However, both CAOS and use of PSI involve additional steps in medical journey of care and increase associated costs.

More recently, use of Virtual Reality (VR) technology has become increasingly popular in orthopaedics for preoperative planning, intraoperative adjuncts and surgical simulations for training surgeons.8 VR technology was developed in the 1980s and consists of a combination of technologies allowing for creation of a simulated and interactive environment. VR places the user within an immersed virtual environment using a head-mounted display (HMD) and a set of sensors allowing for position tracking. Augmented Reality (AR) is a subcategory of VR that overlays virtual graphics over real-life images in real time. AR can involve use of a HMD or not (e.g. via mobile applications). Mixed Reality (MR) is located between the scope of VR and AR where virtual objects are able to interact in real-time with the user and/or its real-world environment.

Recent literature demonstrates promising results regarding the use of VR and AR technology in shoulder arthroscopy to train orthopaedic surgeons.2,9 In 2018, Gregory et al. performed the first RSA combined with MR visualisation using the Microsoft HoloLens (Redmond, Washington, USA) headset. The initial report concluded that real-time intraoperative visualisation of 3D holograms of the patient anatomy preoperatively planned (bones and critical organs such as nerves and arteries) improved the accuracy of the procedure without compromising the patient’s safety.4

Current industry standards

Various preoperative planning software for shoulder arthroplasty are currently commercially available. Registration of the preoperative CT scans onto these platforms allows the surgeons to visualise 3D models of the patient’s anatomy and to perform 3D templating for the creation of PSI guides. Preoperative planning software for shoulder arthroplasty include (but are not limited to) DePuy Trumatch Personalized Solutions System (Warsaw, Indiana, USA), Zimmer Biomet PSI Shoulder for Trabecular Metal Reverse Glenoid System (Warsaw, Indiana, USA), Stryker TrueSight Personalized Planning system (Kalamazoo, Michigan, USA), Arthrex Virtual Implant Positioning System (Naples Florida, USA) and the Wright Tornier Blueprint planning software (Memphis, Tennessee, USA).

Wright Medical (Memphis, Tennessee, USA) recently introduced their Blueprint mixed reality technology with the first use of the system for a shoulder arthroplasty in July 2020 at the Mayo Clinic (Rochester, Minnesota, USA) using the HoloLens2 (Microsoft, Redmond, Washington, USA).3 The addition of MR capabilities to Blueprint planning software will allow the surgeon to visualise 3D holograms of the preoperative plan overlaid onto the surgical field intraoperatively in real-time (Fig.1). This technology also presents the potential to guide the surgeon intraoperatively for implant positioning as a built-in navigation system. This would allow the surgeon to reproduce the preoperatively planned implant position without use of PSI or external computer navigation system. Control and interaction with the 3D holograms using the HoloLens2 headset (Microsoft, Redmond, Washington, USA) is made possible by oral command or simple hand gestures. More work is yet to come to validate this technology in a clinical setting for broad commercialisation.

Fig. 1: HoloBluePrint planning software (Wright Medical, Memphis, Tennessee, USA) using HoloLens 2 headset (Microsoft, Rochester, Minnesota, USA)

Mixed reality solutions for sports shoulder injuries and shoulder arthroplasty

Complex shoulder deformities are not limited to arthroplasty. Glenohumeral instability, trauma and revision arthroplasty pose a spectrum of intraoperative challenges which MR technology can help address. Akunah (Queensland, Australia) provides personalised MR solutions for customised pre-planning with MR technology for the treatment of shoulder instability and revision arthroplasty.

Customised humeral allograft forlarge Hill Sachs lesion combined with mixed reality technology

A 19-year-old patient presented with a first-time traumatic left shoulder dislocation. Analysis of CT scans revealed a large Hill Sachs lesion requiring surgery to prevent recurrent shoulder instability. Akunah’s team of engineers collaborated with the orthopaedic surgeon (AG) to create 3D models of the patient’s anatomy and to plan preoperatively the size and shape of a customised patient-specific graft (Fig.2).

Fig. 2: Preoperative planning for customised humeral head allograft for large Hill Sachs lesion:
a) posterior view of the left humeral head demonstrating the Hill Sachs defect, b) restoration of the humeral head sphericity using a computerised customised graft, c) pre-planned graft dimensions

A size-matched humeral allograft was used. Wearing the HoloLens2 headset, the surgeon was able to visualise the pre-planned 3D holograms in real-time during surgery. This allowed for real-time visualisation of the humeral defect, and graft shape and dimensions (Fig.3 and 4).

Use of preoperative patient-specific planning enabled to craft a personalised graft whilst the hologram was overlayed on the donor allograft. Real-time visualisation of 3D holograms in the operating room allowed the surgeon to match the pre-planned shape of the allograft without requiring 3D printing.

Fig. 3: Intraoperative visualisation of 3D pre-planned holograms for patient-specific humeral allograft

Fig. 4: Intraoperative visualisation of pre-planned 3D holograms (a) and matched surgical execution (b)

Single stage revision RSA with significant glenoid bone loss using mixed reality technology

Revisions of RSA are particularly challenging surgical procedures. Removal of the primary implant is often associated with significant glenoid bone loss. Complex bone defects and joint line medialisation necessitate customised grafts or metallic augments. However, presence of the primary metallic prosthesis in the preoperative CT scans creates significant artefact, which prevents evaluation of the size and extent of the glenoid bone defect preoperatively (Fig.5a). To our knowledge, no commercially available shoulder planning software is able to plan revision cases with metal in situ. The surgeon is therefore left to assess the glenoid anatomy and extent of the defect during surgery. In cases of significant and complex glenoid bone defects a 2-stage procedure may be recommended. The first stage allows for primary implant removal, assessment and perhaps grafting of the glenoid bone defect and implantation of a cement spacer (if required) before the second stage. Postoperative CT images (with the implants now removed) help guide the second stage of the arthroplasty. 2-stage procedures, although the standard of care, do induce significant morbidity to the patient and increase the risks of complications during re-surgery.

Personalised metal artefact reduction CT segmentation (Akunah, Queensland, Australia) enabled the surgeon to achieve a single stage revision RSA by combining preoperative planning and intraoperative visualisations using MR.

The main objective in the preoperative planning phase was to create 3D models of the joint and to subtract metal artefacts and primary implants to allow the surgeon (AG) to ascertain the amount and extent of glenoid bone loss preoperatively. 3D modelling of the complete scapula and collaboration between the surgeon and the engineering team enabled to pre-plan the guide-pin trajectory to maximise reliance on remaining viable bone stock (Fig.5b).

Fig. 5: Preoperative axial CT scan with significant metal artefacts (a), corresponding 3D model with Guidewire trajectory pre-planned at Akunah (Queensland, Australia)

Use of MR headset allowed the surgeon to visualise a 3D hologram of the glenoid and corresponding guidewire intraoperatively, which assisted in baseplate and screws positioning (Fig.6).

Research and Development (Akunah, Queensland, Australia) allows for CT scan segmentation and metal artefact reduction. However, in the past, achieving this single stage surgical execution would have required 3D printing of the scapular 3D model and several attempts in the lab to achieve correct and reliable implant positioning. Finally, no additional surgical time or change in the traditional surgical workflow was noted during these cases.

Combination of preoperative planning and MR technology use provided patient-specific solutions to achieve complex Hill Sachs lesion grafting using humeral allograft and single stage revision RSA with significant glenoid bone loss.

Fig. 6: Intraoperative view of the glenoid bone defect with Guidewire matching 3D hologram

Summary and future perspectives

Immersive technologies have a promising future in the field of surgery at large. Visualisation and interaction with 3D holographic models will assist surgeons to offer more personalised medical care and solutions.

These technologies also have the potential to improve surgical education and training. A recent study evaluating the validity and efficacy of immersive VR (imVR) training in orthopaedic resident education showed that the imVR group completed complex glenoid exposure significantly faster than the control group and with higher instruments handling scores.7 Immersive tools such as VR and MR could also promote virtual “surgeon-to-surgeon” visits allowing for remote mentoring and sharing of skills and techniques. The recent Covid-19 pandemic created a significant shift of paradigm with introduction of social distancing measures and strict travel restrictions. These sudden changes could further encourage use of immersive technologies as a new way for surgeons to meet in a virtual operating room while being continents apart.

In addition, immersive technologies could also be used for training scrub nurses and medical device sales team workflow for enhanced understanding of surgical procedures.

However, challenges remain before broad clinical use of the technology. Research is still on going to validate the technology safe clinical use. Extensive work will also be required to assist in the adoption of such technology by medical teams at large. Adoption will strongly be dependent on cost, availability and ability and simplicity of implementation in the overall pipeline of medical processes and surgical workflows.

1 AOANJRR: National joint replacement registry – hip, knee and shoulder arthroplasty: 2019 annual report. 2 Bartlett JD et al.: Does virtual reality simulation have a role in training trauma and orthopaedic surgeons? Bone Jt J 2018; 100B(5): 559-65 3 Dewey JD: Press Release: Wright medical group N.V. announces first shoulder arthroplasty procedure using BLUEPRINTTM mixed reality technology. Published 2020. 4 Gregory T et al.: Surgery guided by mixed reality: presentation of a proof of concept. Acta Orthop 2018; 89(5): 480-3 5 Gupta A et al.: Management of glenoid bone defects with reverse shoulder arthroplasty – surgical technique and clinical outcomes. J Shoulder Elb Surg 2018; 27(5): 853-62 6 Kircher Jet al.: Improved accuracy of glenoid positioning in total shoulder arthroplasty with intraoperative navigation: A prospective-randomized clinical study. J Shoulder Elb Surg 2009; 18: 515-20 7 Lohre Ret al.: Improved complex skill acquisition by immersive virtual reality training. J Bone Jt Surg Am 2020; 102-A(6) 8 Lohre R et al.: The evolution of virtual reality in shoulder and elbow surgery. JSES Int 2020; 4(2): 215-23 9 Ponce BA et al.: Telementoring: use of augmented reality in orthopaedic education: AAOS exhibit selection. J Bone Jt Surg Am 2014; 96(10): e84(1) 10 Villatte G et al.: Use of patient-specific instrumentation (PSI) for glenoid component positioning in shoulder arthroplasty. A systematic review and meta-analysis. PLoS One 2018; 13(8): e0201759

Back to top