Abstract
Background/Aim: To develop and evaluate the accuracy of augmented reality (AR)-based patient positioning systems in radiotherapy. Materials and Methods: AR head-mounted displays (AR-HMDs), which virtually superimpose a three-dimensional (3D) image generated by the digital imaging and communications in medicine (DICOM) data, have been developed. The AR-based positioning feasibility was evaluated. Then, the setup errors of three translational axes directions and rotation angles between the AR and the conventional laser-based positioning were compared. Results: The AR-based pelvic phantom positioning was feasible. The setup errors of AR-based positioning were comparable to laser-based positioning in all translational axis directions and rotation angles. The time necessary for AR-based positioning was significantly longer than that for laser-based positioning (171.0 s vs. 47.5 s, p<0.001). Conclusion: AR-based positioning for radiotherapy was feasible, and showed comparable positioning errors to those of conventional line-based positioning; however, a markedly longer setup time was necessary.
Recently, demand for radiotherapy has increased to meet the needs of the growing number of cancer patients (1-4). To perform accurate radiotherapy, accurate and reproducible patient positioning on the couch of a linear accelerator (LINAC) is fundamentally important. Traditionally, patient positioning for radiotherapy has been performed by positioning laser and alignment markers drawn on a patient’s skin (5-7). For image-guided radiotherapy (IGRT), patient positioning is performed using cone-beam computed tomography (CBCT) and planar kilovolt (kV) images. However, alignment markers drawn on the patient body surface are still necessary for body position adjustment. Such patient positioning methods have some limitations. Cosmetic issues caused by alignment markers represent one limitation. Moreover, alignment markers must be maintained during the treatment period, leading to deterioration of the patient’s quality of life (8-10).
Augmented reality (AR), a variant of virtual reality (VR), enhances real-world imaging by virtually superimposing computer-generated images (11-14). If AR is applied to the patient position at the time of radiotherapy, then it might become possible to skip line drawing on the patient’s body.
For this study, an AR-based patient positioning system was developed using a commercially available software and a wearable device. In addition, the accuracy of patient positioning using the AR-based patient positioning system was confirmed by comparison with conventional laser-based positioning.
Materials and Methods
This study, because it involved a phantom experiment, did not require the approval of an institutional review board (IRB).
For this study, the AR head-mounted display (AR-HMD) system, which superimposes a three-dimensional (3D)-image generated using digital imaging and communications in medicine (DICOM) data was developed to enable positioning without drawing of alignment markers. Then, the setup errors of the developed AR-based positioning were compared using the AR-HMD system and conventional laser-based positioning. All experiments were performed using a pelvic phantom (BrainLab pelvis phantom; BrainLab Medical Systems, Westchester, IL, USA) that includes an iron ball (Figure 1). This iron ball was used as an isocenter for this study.
Development of the AR-HMD system. The AR-HMD system development process consists of the following five steps as presented in Figure 2. First, DICOM data of the CT imaging of pelvic phantom were acquired using a CT scanner (SOMATOM Definition AS; Siemens Healthineers, Forchheim, Germany) (Step 1). Second, DICOM data of the pelvic phantom CT images were transferred to the image workstation (Ziostation2; Ziosoft Inc., Tokyo, Japan) to generate 3D images of the surface structure and iron ball of the pelvic phantom (Step 2). The created 3D-CT DICOM data were converted to the OBJ file format and imported to the game development engine software (Unity; Unity Technologies, San Francisco, CA, USA). Using this software, the color, size, and transparency of the 3D pelvic phantom image were adjusted (Figure 3A). In addition, three virtual lasers (x, y, and z axis) were added at the isocenter (i.e., position of the iron marker) of the 3D pelvic phantom image (Figure 3A). This virtual laser was used at the time of positioning of the pelvic phantom on the LINAC couch with a six-axis drive (TrueBeam; Varian Medical System, Palo Alto, CA, USA), as described in a later section.
Next, an application that enables the position adjustment of a 3D phantom image was created using game development engine software (Unity) and tool kit software (Mixed Reality ToolKit; Microsoft Corp., Redmond, WA, USA) (Step 3). This position adjustment application includes two functions: a hand tracking function that can track hand movements and which can manipulate the 3D phantom image position and a dedicated control panel function that enables fine adjustment of the 3D phantom image position (Figure 3A).
Then, in step 4, both the 3D pelvic phantom image generated in step 2 and the position adjustment application developed in step 3 were incorporated into an executable file using the integrated application development environment (Microsoft Visual Studio; Microsoft Corp.). The application was transferred to a wearable mixed reality device (HoloLens 2; Microsoft Corp.) (Figure 3B). Finally, both the hologram of the 3D pelvic phantom image and real-world objects were displayed simultaneously on the wearable mixed reality device (Step 5) (Figure 3C and 3D).
AR-based and laser-based positioning. After AR-HMD system development, the feasibility of positioning the pelvic phantom on the LINAC couch using the AR-HMD system (AR-based positioning) was evaluated. Subsequently, conventional laser-based positioning was also performed. Their setup errors and setup times were compared. Workflows of AR-based and laser-based positioning methods are portrayed in Figure 4.
In AR-based positioning, an operator wore the AR-HMD system, which displays both the 3D pelvic phantom image and three virtual lasers (Figure 3C). Then, using the hand tracking function and the dedicated control panel function, the location of the virtual isocenter of the 3D phantom image was adjusted at the location of the mechanical isocenter of LINAC indicated by the positioning lasers (Figure 3D and 4A). Here, three virtual lasers were used as a guide to adjust the 3D phantom image virtual isocenter location and the mechanical isocenter of LINAC. After adjustment of the 3D phantom image, the real pelvic phantom was set on the LINAC couch so that the 3D phantom image and the real pelvic phantom were completely overlapping using the AR-HMD system (Figure 4A). In AR-based positioning, alignment markers were not drawn on the surface of the real pelvic phantom.
Using laser-based positioning, pelvic phantoms were created with three surface alignment markers respectively showing the x, y, and z axis of the isocenter (Figure 4B). Using the alignment marker of the pelvic phantom and the positioning lasers of LINAC, the respective isocenter locations were adjusted.
Assessments. Three radiation therapy technologists, respectively with 22, 17, and 13 years of experience in radiotherapy, performed AR-based positioning 10 times each (total 30 times). The feasibility was evaluated based on their evaluations. Feasibility of AR-based positioning was defined as successful placement of the pelvic phantom on the couch of LINAC using the AR-HMD system. Subsequently, conventional laser-based positioning was also performed by the same radiological technologist 10 times each. Setup errors between the AR-based and laser-based positioning were compared. For this study, the setup error was defined as the difference between the pre-planned phantom position and the actual phantom position set by AR-based or laser-based positioning. Three translational errors (i.e., vertical, longitudinal, and lateral) and three rotational errors (i.e., pitch, roll, and yaw) were evaluated as setup errors. The errors of each setup were measured using image registration software (Varian Offline Review software; Varian Medical System, Palo Alto, CA, USA). The time necessary for each positioning method was also measured.
Statistical analysis. The setup errors of AR-based and laser-based positioning were analyzed using a linear mixed-effects model for repeated measures (15). For each of the setup error and time measurements, the least squares (LS) means and 95% confidence interval (CI) were calculated for each group based on the mixed-effects model. The LS mean difference between AR-based and laser-based positioning for each of the measurements and its 95%CI were calculated. Contrast tests for LS mean differences were also performed. The confidence intervals and p-values were calculated based on the sandwich-type robust standard error. A two-sided p-value of less than 0.05 was regarded as statistically significant without adjustment for multiplicity. All statistical analyses were conducted using SAS software (SAS Institute Inc., Cary, NC, USA).
Results
Feasibility. The pelvic phantom was placed on the couch of LINAC in all AR-based positionings. Therefore, the feasibility of AR-based positioning was 100% (30/30).
Setup error. The LS means of the translational errors in AR-based and laser-based positioning were –0.82 mm (95%CI=–2.89-1.25 mm) and 0.10 mm (95%CI=–0.18-0.38 mm) on the vertical axis, 0.27 mm (95%CI=–0.47-1.01 mm) and 0.10 mm (95%CI=–0.14-0.35 mm) on the longitudinal axis, and –0.58 mm (95%CI=–3.22-2.07 mm) and 0.07 mm (95%CI=–0.31-0.44 mm) on the lateral axis (Figure 5A). The LS mean differences between AR-based and laser-based positioning for the vertical, longitudinal, and lateral axis were, respectively, –0.92 mm (95%CI=–3.23-1.40 mm) (p=0.333), 0.17 mm (95%CI=–0.57-0.90 mm) (p=0.562), and –0.64 mm (95%CI=–3.50-2.22 mm) (p=0.566).
The LS means of the rotational errors in AR-based and laser-based positioning were 0.21° (95%CI=–0.12-0.53°) and 0.00° (95%CI=–0.04-0.05°) for yaw, 0.22° (95%CI=0.18-0.26°) and 0.18° (95%CI=0.16-0.21°) for pitch, and 0.03° (95%CI=0.02-0.04°) and 0.04° (95%CI=–0.01-0.09) for roll (Figure 5B). The LS mean differences between AR-based and laser-based positioning for yaw, pitch, and roll were, respectively, 0.20° (95%CI=–0.10-0.50°) (p=0.136), 0.03° (95%CI=–0.03-0.10°) (p=0.231), and –0.01° (95%CI=–0.06-0.05°) (p=0.744).
Setup time. The LS means of the setup time in AR-based and laser-based positioning were, respectively, 171.0 s (95%CI=135-206.9 s) and 47.5 s (95%CI=43.9-51.2 s) (Figure 5C). The LS mean difference between AR-based and laser-based positioning was 123.4 s (95%CI=88.7-158.2 s). Significant difference was found between the time necessary for the AR-based and laser-based setup (p=0.001). Results show that AR-based setup requires more setup time than laser-based setup.
Discussion
Conventional line-based patient positioning, which uses alignment markers drawn on the patient’s body surface, is useful for accurate and reproducible patient positioning during radiotherapy. However, such alignment markers induce cosmetic difficulties and degrade the patients’ quality of life. Therefore, a positioning method that requires no drawing of surface alignment markers has been warranted.
Results of this study demonstrated that the newly developed AR-HMD system enabled performance of positioning for radiotherapy without drawing alignment markers on the surface. Moreover, translational and rotation errors between AR-based and laser-based systems were comparable in this phantom experiment. A tendency remained by which positioning errors in the AR-based positioning were greater than those of line-based positioning. This tendency might be attributable to the difference in the number of position adjustments between AR-based and line-based positioning. For line-based positioning, position adjustment was necessary only once: for position adjustment of the alignment marker on the pelvic phantom and for positioning of lasers of LINAC. However, for AR-based positioning, position adjustment was necessary twice: once for adjustment between the virtual and mechanical isocenter and once for adjustment between the 3D virtual phantom and real pelvic phantom. Another possible explanation for the larger positioning errors in the AR-based positioning is the technical limitations in the hologram stability of the AR-HMD system used for this study (16-18). Vassallo et al. reported the displacement error of the HoloLens 2 as 5.83±0.51 mm (19).
The setup time necessary for AR-based positioning was significantly longer than that for laser-based positioning. A possible reason for this finding is the difference in the number of position adjustments between AR-based and line-based positioning. Additional position adjustment in AR-based positioning might lead to the longer setup time.
Although AR-based positioning required considerably more setup time and although it showed a tendency to have larger setup error, these drawbacks can be overcome through further development. For example, if a software that automatically adjusts the virtual and mechanical isocenter was developed, then it would be possible to reduce the position adjustment error and setup time. An improvement of the holographic software in regards to hologram stability is also expected to be helpful to increase the AR-based positioning accuracy.
This study includes several limitations. As an experimental phantom study performed using only a pelvic phantom, this study cannot provide results that are directly applicable to clinical practice. Further study conducted with phantoms of other types as well as in vivo experiments must be conducted to evaluate the utility of AR-based positioning. The small sample size examined in this study is another major limitation.
Despite these limitations, results obtained through this study suggest that the AR-HMD system is useful for positioning for radiotherapy without drawing alignment markers on the surface. Results of this study also provide useful information to increase the positioning accuracy and to shorten the setup time of AR-based positioning for radiotherapy.
Conclusion
AR-based positioning for radiotherapy was demonstrated as feasible in this phantom study. AR-based positioning showed comparable positioning error to that of conventional line-based positioning, although significantly longer setup time was necessary.
Acknowledgements
This research was conducted with the support of Hyogo College of Medicine, Hyogo, Japan and the support of research funding to promote the hospital function of the Japan Industrial Safety and Health Organization.
Footnotes
This article is freely accessible online.
Authors’ Contributions
KT conceived the study, collected and analysed the data, and contributed to writing the manuscript. HT was responsible for the study conception and design, and conducted the study and drafting of the article. AO, FH, YT, SN performed the experiments. MI analysed the data. MF, KK, MT revised the article. HT, KY administrated and supervised the study. All Authors read and approved the final manuscript.
Conflicts of Interest
No actual or potential conflicts of interest exist in relation to this study.
- Received April 18, 2021.
- Revision received May 27, 2021.
- Accepted May 28, 2021.
- Copyright © 2021 International Institute of Anticancer Research (Dr. George J. Delinasios), All rights reserved