Abstract

Path planning algorithms for robotics can be as simple as having an operator program waypoints into a robot's controller and having the robot perform a simple task such as welding. This works well in an industrial setting but will not work for complicated tasks such as performing surgery. Another approach would be to use a constraint function in programing a robot to perform surgery, but it would be difficult to capture and represent all of the surgeon's information in the mathematical terms required for a cost function. A third approach, and the one utilized in this study, is to train a set of artificial neural networks (ANNs) using recorded surgeons' motions when manipulating a surgical instrument during procedure training using a surgery simulator. This has the advantage of indirectly capturing the surgeon's abilities and intentions without needing to explicitly capture all of the motion information that must be encoded from their trajectory planning and decision-making, and then, say, creating a complex constraint function using that information. In this research effort, virtually captured surgical trajectories from trained surgeons were used to train ANNs, after being preprocessed into three subtasks. Each set of subtask data was used to train a separate ANN. Each of the ANNs was trained using a custom cost function and evaluated using custom metrics. During the training, the positions of fiducial markers, recorded during procedure attempts, were used to orient the recorded path relative to the patient's anatomy. Although the ANN-generated trajectories were not used to perform surgery on a live patient in this study, the fiducial marker position information is intended to be exploited during a real procedure to position, orient, and scale a tool trajectory to suit a patient's specific anatomy. The trained ANNs were subjected to several tests to assess their safety and robustness. We found that even when trained on a small number of datasets, the ANNs converged and could generate output trajectories that were still assessed to be safe even when slight changes in the fiducial marker placement locations were given.

References

1.
Stanford Medicine
,
2020
, “
General Surgery
,” Stanford Medicine,
Stanford, CA
, accessed July 12, 2020, https://stanfordhealthcare.org/medical-clinics/surgery-clinic/patient-resources/surgery-statistics.html
2.
National Institute on Deafness and Other Communication Disorders
,
2017
, “
Cochlear Implants
,” National Institute on Deafness and Other Communication Disorders,
Bethesda, MD
, accessed July 12, 2020, https://www.nidcd.nih.gov/health/cochlear-implants#:~:text=In%20the%20United%20States%2C%20roughly%2058%2C000%20devices%20have,Administration%20%5BFDA%5D%2C%20as%20reported%20by%20cochlear%20implant%20manufacturers.%29
3.
Davies
,
B.
,
2000
, “
A Review of Robotics in Surgery
,”
Proc. Inst. Mech., Part H
,
214
(
1
), pp.
129
140
.10.1243/0954411001535309
4.
Badaan
,
A. R.
, and
Stoianovici
,
D.
,
2011
,
Robotic Systems: Past, Present, and Future, in Robotics
,
A. K.
,
Hemal
, and
M.
,
Menon
, ed.,
Springer
,
London
, pp.
655
665
.
5.
Biomedhealthtech
,
2015
, “
The ROBODOC Surgical System
,”
Biomedhealthtech
,
Marthanda, Mumbai, India
, accessed July 12, 2020, http://www.biomedhealthtech.com/robodoc.html
6.
Danilchenko
,
A.
,
2011
, “
Fiducial-Based Registration With Anisotropoc Localization Error
,” Ph.D. thesis,
Vanderbilt University
,
Nashville, TN
, p.
170
.http://hdl.handle.net/1803/12140
7.
Lipschitz
,
N.
,
2019
, “
Surgical Resident Training in Otolaryngology
,” Personal Interview.
8.
Mayer
,
H.
,
Gomex
,
F.
,
Wierstra
,
D.
,
Nagy
,
I.
,
Knoll
,
A.
, and
Schmidhuber
,
J.
,
2006
, “
A System for Robotic Heart Surgery That Learns to Tie Knots Using Recurrent Neural Networks
,”
IEEE/RSJ International Conference on Intelligent Robotics and Systems
,
Beijing, China
, Oct. 9–15, pp.
543
548
.10.1109/IROS.2006.282190
9.
Moustris
,
G. P.
,
Hiridis
,
S. C.
,
Deliparaschos
,
K. M.
, and
Konstantinidis
,
K. M.
,
2011
, “
Evolution of Autonomous and Semi-Autonomous Robotic Surgical Systems: A Review of the Literature
,”
Int. J. Med. Rob. Comput. Assisted Surg.
,
7
(
4
), pp.
375
392
.10.1002/rcs.408
10.
Mayer
,
H.
,
Nagy
,
I.
,
Knoll
,
A.
, and
Schirmbeck
,
E. U.
,
2004
, “
The Endo[PA]R System for Minimally Invasive Robotic Surgery
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Sendai, Japan
, Sept. 28–Oct. 2, pp.
3637
3642
.10.1109/IROS.2004.1389980
11.
Korte
,
C.
,
2020
, “
A Preliminary Investigation Into Using Artificial Neural Networks to Generate Surgical Trajectories to Enable Semi-Autonomous Surgery in Space
,” Ph.D. thesis,
University of Cincinnati
,
Cincinnati, OH
.https://search.proquest.com/openview/ddc48e0b8270a5428020b4e6f9c66ebc/1?pq-origsite=gscholar&cbl=18750&diss=y
12.
Wiet
,
G. J.
,
Stredney
,
D.
,
Kerwin
,
T.
,
Hittle
,
B.
,
Fernandez
,
S. A.
,
Abdel-Rasoul
,
M.
, and
Welling
,
D. B.
,
2012
, “
Virtual Temporal Bone Dissection System: OSU Virtual Temporal Bone System: Development and Testing
,”
Laryngoscope
,
122
(
Suppl. 1
),
S1
12
.10.1002/lary.22499
13.
Kerwin
,
T.
,
Hittle
,
B.
,
Wiet
,
G. J.
, and
Stredney
,
D.
,
2001
, “
Virtual Simulation of Temporal Bone Dissection
,”
Ohio Supercomputing
,
Columbus, OH
.
14.
Ohio Supercomputing Center,
2017
, “
Virtual Simulations Prep Future Doctors for Surgery
,”
Ohio Supercomputing Center
,
Columbus, OH
,
Report No. 2008
.https://www.osc.edu/research/research-reports/2008/virtual_simulations_prep_future_doctors_for_surgery
15.
Goodfellow
,
I.
,
Bengio
,
Y.
, and
Courville
,
A.
,
2016
, “
Sequence Modeling: Recurrent and Recursive Nets
,”
Deep Learning
,
MIT Press
,
Cambridge, MA
, p.
399
.https://www.deeplearningbook.org/slides/10_rnn.pdf
16.
Hochreiter
,
S.
, and
Schmidhuber
,
J.
,
1997
, “
Long Short-Term Memory
,”
Neural Comput.
,
9
(
8
), pp.
1735
1780
.10.1162/neco.1997.9.8.1735
17.
Chollet
,
F.
,
2015
, Keras, GitHub, San Francisco, CA.
18.
Abadi
,
M.
,
Agarwal
,
A.
,
Barham
,
P.
,
Brevdo
,
E.
,
Chen
,
Z.
,
Citro
,
C.
,
Corrado
,
G. S.
,
Davis
,
A.
,
Dean
,
J.
,
Devin
,
M.
,
Ghemawat
,
S.
,
Goodfellow
,
I.
,
Harp
,
A.
,
Irving
,
G.
,
Isard
,
M.
,
Jozefowicz
,
R.
,
Jia
,
Y.
,
Kaiser
,
L.
,
Kudlur
,
M.
,
Levenberg
,
J.
,
Mané
,
D.
,
Schuster
,
M.
,
Monga
,
R.
,
Moore
,
S.
,
Murray
,
D.
,
Olah
,
C.
,
Shlens
,
J.
,
Steiner
,
B.
,
Sutskever
,
I.
,
Talwar
,
K.
,
Tucker
,
P.
,
Vanhoucke
,
V.
,
Vasudevan
,
V.
,
Viégas
,
F.
,
Vinyals
,
O.
,
Warden
,
P.
,
Wattenberg
,
M.
,
Wicke
,
M.
,
Yu
,
Y.
, and
Zheng
,
X.
,
2015
, “
Tensorflow: Large-Scale Machine Learning on Heterogeneous Systems
,”
Proceedings of the 12th USENIX Symposiom on Operating Systems Design and Implementation
,
Savannah, GA
, Oct., pp.
265
283
.https://arxiv.org/abs/1603.04467
You do not currently have access to this content.