Abstract
Background/Aim: The aim of this phantom study was to evaluate the learning curves of novices practicing how to place a cone-beam computed tomography (CBCT)-guided needle using a novel robotic assistance system (RAS). Materials and Methods: Ten participants performed 18 punctures each with random trajectories in a phantom setting, supported by a RAS over 3 days. Precision, duration of the total intervention, duration of the needle placement, autonomy, and confidence of the participants were measured, displaying possible learning curves. Results: No statistically significant differences were observed in terms of needle tip deviation during the trial days (mean deviation day 1: 2.82 mm; day 3: 3.07 mm; p=0.7056). During the trial days, the duration of the total intervention (mean duration: day 1: 11:22 min; day 3: 07:39 min; p<0.0001) and the duration of the needle placement decreased (mean duration: day 1: 03:17 min; day 3: 02:11 min; p<0.0001). In addition, autonomy (mean percentage of achievable points: day 1: 94%; day 3: 99%; p<0.0001) and confidence of the participants (mean percentage of achievable points: day 1: 78%; day 3: 91%; p<0.0001) increased significantly during the trial days. Conclusion: The participants were already able to carry out the intervention precisely using the RAS on the first day of the trial. Throughout the trial, the participants’ performance improved in terms of duration and confidence.
Image-guided minimally invasive procedures are increasing in modern medicine (1). C-arm-based guidance together with cone-beam computed tomography (CBCT) imaging offer different possibilities in comparison to conventional CT-guided interventions: absence of a gantry and visualization of the complete needle track in off-plane punctures by 2D imaging (2).
Nevertheless, the result of needle puncture still depends on individual experience as needles are placed completely manually (3). For standardization and better planning of interventions, various assistance systems have been introduced in recent years (4-7). Several studies focused on precision, radiation exposure, and possibly reducing time (8-10). However, the possible benefits of these various systems for training inexperienced interventionalists have not been sufficiently examined.
The aim of this study, therefore, was to assess and compare the performance of novices using a robotic assistance system (RAS) and compare the results over several days.
Materials and Methods
Ten medical students were recruited as participants to perform the intervention using a RAS. Before their first puncture, each participant received a 90-min individual introduction to the system based on Peyton’s 4-step approach (11). Afterwards, each participant performed 18 punctures over the course of 3 days, split into six punctures per day. A commercial abdominal phantom (Model 057A, Computerized Imaging Reference Systems Inc.; Norfolk, VA, USA) with various lesions was employed. The individual lesions to be punctured and the puncture angle were predetermined by a supervisor.
During imaging, the participant and the supervisor left the operating room to prevent any radiation exposure. No institutional review board approval was necessary due to the phantom setting and zero radiation exposure. This work was in part funded by the German Federal Ministry of Research (BMBF) as part of the M2OLIE Mannheim Research Campus initiative (Forschungscampus).
Interventional setting and image acquisition. For imaging with a C-arm system (Artis Zeego, Siemens Healthcare GmbH, Munich, Germany) and the possibility of using cone-beam CT (CBCT), a dedicated operating table (Maquet Magnus, Getinge AB, Getinge, Sweden), a control unit for the table, and a C-arm were used. The image reconstructions were performed using a dedicated workstation (Syngo Workstation, Siemens Healthcare GmbH).
The C-arm CT used a 200° rotation in 6 s, with image acquisition every 0.5°. The reconstructed field of view was 250 mm in diameter and 190 mm in height, divided into 397 images with an image matrix of 512×512 pixel. The tube voltage was 90 kV with a pulse time of 3.5 ms. No collimation was used.
In all test runs, the RAS was located on the left side of the operating table and was already prepared for the intervention before starting the test run. Booting the RAS in the intervention room was therefore not part of the test runs.
Robotic assistance system. The RAS consisted of a lightweight robot (LBR iiwa 14 R820, KUKA AG, Augsburg, Germany) mounted on a mobile base. The setup is depicted in Figure 1. The RAS featured a dedicated, prototype navigation software, which can receive medical images using the DICOM format, detecting the calibration spheres enclosed inside the corpus at the distal end of the robotic arm, virtual planning of needle paths, and validating the intraprocedural needle path using 2D imaging. Further details are described by Kostrzewa et al. (8). Figure 2 shows the workflow in detail.
Depiction of the robotic assistance system: mobile platform and robotic assistance system (A). Placement of needle with robotic assistant (B).
Workflow during the intervention. CBCT: Cone beam computed tomography.
Measured variables. Precision was measured in two ways. The coordinates of the needle tip and the target point were tracked in 3D in the control CBCT using the DICOM viewer of the workstation (Syngo Workstation, Siemens Healthcare GmbH). Then, the deviation between these coordinates was calculated. Additionally, the number of iterations, defined as repositioning of the needle, were documented by the supervisor during the experiments.
In agreement with the participants, the interventions were recorded by cameras mounted in the operating room. The timestamps of the video recordings were used to determine the duration of the total intervention and the duration of the actual needle placement. The duration of the total intervention was measured from the start of selecting the protocol for planning the CBCT scan to the final CBCT scan after finishing the needle placement. The duration of the needle placement was measured from the robotic movement to the desired guidance position until the final 2D validation imaging of the needle path (Figure 2).
A questionnaire based on the ‘objective structured assessment of technical skills’ (12) was developed to determine the autonomy and confidence of the participants during the intervention. To assess autonomy, the supervisor documented the intervention according to a checklist of the individual steps. A point was awarded for each step the participant carried out independently. Half a point was awarded for minor questions to the supervisor and zero points for intensive assistance. The maximum number of points to be achieved was 17 and the score is depicted as a percentage. Confidence was assessed by a questionnaire (Figure 3) consisting of 9 items with a 6-point Likert scale (1=unsure; 6=confident) and is also depicted as a percentage. The questionnaire was filled out by the participants after each test run.
Confidence questionnaire.
Statistical analysis. All statistical calculations were accomplished using SAS, release 9.4 (SAS Institute, Cary, NC, USA). The Sidak Test was performed to compare the mean values of different test days. Statistical significance was assumed for p-values less than 0.05. Part of the data is displayed in line charts, generated in MS Office (Microsoft Corporation, Redmond, WA, USA).
Results
Out of 180 test runs, 177 were completed (in plane: 89; out of plane: 88); three test runs were excluded due to technical errors. A summary of the results is given in Table I.
Summary of results presented as: mean±STD (min-max).
A comparison of the mean penetration depths showed no significant differences. There was no statistically significant difference in terms of needle tip deviation between the test days (Figure 4). The mean deviation was 2.82±1.18 mm on day 1, followed by 3.04±1.44 mm on day 2, and 3.07±1.51 mm on day 3. Only one iteration of the needle was recorded throughout the study.
Learning curve: Precision and Duration (mean values).
The mean duration of the total intervention decreased significantly (p<0.0001) from 11:22±03:10 min on day 1 to 08:29±1:40 min on day 2. On day 3, the mean total duration of the intervention was 07:39±00:59 min and showed no significant differences (p=0.0955) when compared to day 2 (Figure 4).
The mean duration of the needle placement was 03:17±01:03 min on trial day 1, 02:33±00:50 min on trial day 2, and 02:11±00:30 min on day 3 and differed significantly between both day 1 and day 2 (p<0.0001) and day 2 and day 3 (p=0.0420).
Already on day 1, the participants achieved a high autonomy with an average of 94%±8% of the achievable points (Figure 5). On day 2, the mean result was 98%±6% and on day 3 99%±2%. Day 1 and day 2 differed significantly (p<0.0001), while day 2 and day 3 showed no significant differences (p=0.4068) in terms of autonomy. Indeed, on day 1 of the trial, 53% of the interventions were already carried out without any assistance.
Learning Curve: Autonomy and Confidence (mean values).
The confidence of the participants increased significantly (Figure 5). The mean score achieved was 78%±12% on day 1, 84%±11% on day 2, and 91%±10% on day 3. Confidence on day 1 and day 2 (p<0.0001), as well as on day 2 and day 3 (p=0.0033) differed significantly.
Discussion
This study evaluated the learning curves of novices performing CBCT-assisted needle placements in a model phantom employing a novel robotic system. Precision, duration of the total intervention, duration of the needle placement, autonomy, and confidence of the unexperienced participants during the interventions were assessed.
The precision that the participants of this study achieved was comparable and consistent (mean deviation day 1: 2.82 mm; day 2: 3.04 mm; day 3: 3.07 mm; p>0.05) throughout the trial days. Furthermore, the precision they achieved is comparable to that for other needle-guidance systems, which show a needle tip deviation of approximately 1 to 3 mm (4, 5, 13) in general, but were operated by experienced interventionalists.
The team of Guiu et al. compared robotic-assisted CT-guided needle placements between three experienced and one inexperienced user on sedated pigs. The two groups showed no significant difference in precision (mean 4 mm vs. 4.3 mm) and are comparable to the precision of the inexperienced participants in the present study (mean 3.8 mm). Guiu et al. also report that iterations occurred in 14% of the interventions, with no significant differences between the experienced and inexperienced users (7).
Kostrzewa et al. observed a similar precision (mean 2.74 mm) without any iteration from an experienced interventionalist using a similar RAS and trial design as in this study (8). The mean duration of the total intervention (6:01 min) by the experienced interventionalists of Kostrzewa et al. was faster than the time needed by the unexperienced participants in this study on the first day, but these times equalized as the training of the participants proceeded (mean total duration day 1: 11:22 min; day 2: 08:29 min; day 3: 07:39 min). Thereby, the total duration decreased by 03:42 min (−32.6%) and the duration of the needle placement by 01:06 min (−33.4%) in the mean over the three trial days in this study.
The duration of the needle placement (mean duration day 1: 03:17 min; day 2: 02:33 min; day 3: 02:11 min; p<0.05) differed significantly across all three trial days, implying a continuous learning curve. This period represents the part of an intervention that is repeated in interventions requiring multiple needle placements, such as microwave ablation, radiofrequency ablation, or irreversible electroporation (14-16). However, it is difficult to compare the duration of the needle placement with other studies due to differences in the setting and/or definition of start and end.
In addition to the decreasing duration of the total intervention, assistance by the supervisor was less and less necessary and the participants were increasingly able to work alone. The participants made the most progress (p<0.0001) in autonomy between the first and second test day (mean day 1: 94%; day 2: 98%; day 3: 99%). Mean confidence increased significantly over all three days (mean day 1: 78%; day 2: 84%; day 3: 91%; p<0.05). These results may influence interventional navigation system usage and how training for interventional procedures is structured (17, 18).
The greatest limitation stems from the phantom setting, e.g., no motion and the haptics of the phantom differ from those for human tissue. Additionally, the knowledge of the participants puncturing a phantom may also encourage them to act more bluntly (19). This may influence the intervention time in particular.
In summary, the data showed that novices were able to place a needle using a RAS precisely and adequately in a phantom setting with CBCT imaging and could even increase efficacy over the course of the 3 trial days.
Footnotes
Authors’ Contributions
PL: Data curation, investigation, methodology, project administration, visualization, writing-original draft. NR: Visualization, writing - review & editing. AR: Conceptualization, funding acquisition, methodology, project administration, resources, software, validation, visualization, writing-review & editing. MK: Methodology, software, validation, writing - review & editing. SS: Resources, writing - review & editing. MN: Resources, writing - review & editing. SH: Formal analysis, writing - review & editing. SOS: Funding acquisition, writing - review & editing. JS: Funding acquisition, writing - review & editing. SD: Conceptualization, funding acquisition, methodology, project administration, resources, supervision, validation, writing - review & editing.
Funding
This work was supported by German Federal Ministry for Education and Research [grant numbers 13GW0090, 13GW0389A]; the European Community’s Seventh Framework Programme (FP7/2007-2013) [grant number 602306], BEC GmbH and the German Federal Minitry of Economic Affairs and Energy [grant number ZF4517701AW7]. None of the funding sources had direct influence on the study design; in the collection, analysis, and interpretation of data; in the writing of the manuscript nor in the decision to submit the article for publication.
Conflicts of Interest
The Institute of Clinical Radiology and Nuclear Medicine has research agreements with Siemens Healthcare GmbH. The Fraunhofer Institute for Manufacturing Engineering and Automation IPA Department for Clinical Health Technologies has research agreements with BEC GmbH. Mr. Rothfuss also reports that, after the completion of the presented work, he was employed by BEC GmbH. Mr. Lautenschlaeger used to be with Fraunhofer IPA till 31.03.2021.
- Received December 30, 2022.
- Revision received January 17, 2023.
- Accepted January 18, 2023.
- Copyright © 2023, International Institute of Anticancer Research (Dr. George J. Delinasios), All rights reserved
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY-NC-ND) 4.0 international license (https://creativecommons.org/licenses/by-nc-nd/4.0).
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.