Keywords
computer vision, artificial neural networks, Baxter, intelligent systems, surgical tool detection
Presentation Type
Event
Research Abstract
Several hospitals face nurse staffing shortages for surgeries. This research focuses on building a system with Baxter robot capable of identifying surgical tools using computer vision and delivering them to the surgeon on demand. This would deal with the issue of nurse unavailability during simple surgical procedures. The key aspects of the project were: testing the accuracies of various Artificial Neural Networks (ANNs) in classifying surgical instruments, and programming Baxter to implement a surgical tool delivery system using magnets at the tip of its 7-DOF robotic arms. The methodology consisted of, first, implementing algorithms to enable Baxter to do pick and deliver tasks for surgical tools, and second, gathering Hu-Moments of various tools using the cameras on Baxter's arm, which were then used to train the ANNs. Tool detection accuracies of ANNs with hidden layer neuron number varying from 5 – 50 and learning rates varying from 0.005 - 0.1 were collected. Then, the tool identification and tool delivery system were merged together to create a turn by turn dynamic tool tracking and delivery system, which retrieved tools, based on the surgeons input, through a Leap Motion sensor. In addition to delivery, the system was modified to retrieve used tools back from the surgeon, using a computer vision based approach. The optimal ANN configuration consisted of an ensemble of various ANNs working together and achieved a detection accuracy of 93%. The average time taken for a mock abdominal incision surgery with the system is expected to be around 10 min and 30 sec.
Session Track
Health
Recommended Citation
Sthitapragyan Parida, Juan Pablo Wachs, and Maria Eugenia Cabrera,
"Dynamic Surgical Tool Tracking and Delivery System using Baxter Robot"
(August 7, 2014).
The Summer Undergraduate Research Fellowship (SURF) Symposium.
Paper 51.
https://docs.lib.purdue.edu/surf/2014/presentations/51
Included in
Dynamic Surgical Tool Tracking and Delivery System using Baxter Robot
Several hospitals face nurse staffing shortages for surgeries. This research focuses on building a system with Baxter robot capable of identifying surgical tools using computer vision and delivering them to the surgeon on demand. This would deal with the issue of nurse unavailability during simple surgical procedures. The key aspects of the project were: testing the accuracies of various Artificial Neural Networks (ANNs) in classifying surgical instruments, and programming Baxter to implement a surgical tool delivery system using magnets at the tip of its 7-DOF robotic arms. The methodology consisted of, first, implementing algorithms to enable Baxter to do pick and deliver tasks for surgical tools, and second, gathering Hu-Moments of various tools using the cameras on Baxter's arm, which were then used to train the ANNs. Tool detection accuracies of ANNs with hidden layer neuron number varying from 5 – 50 and learning rates varying from 0.005 - 0.1 were collected. Then, the tool identification and tool delivery system were merged together to create a turn by turn dynamic tool tracking and delivery system, which retrieved tools, based on the surgeons input, through a Leap Motion sensor. In addition to delivery, the system was modified to retrieve used tools back from the surgeon, using a computer vision based approach. The optimal ANN configuration consisted of an ensemble of various ANNs working together and achieved a detection accuracy of 93%. The average time taken for a mock abdominal incision surgery with the system is expected to be around 10 min and 30 sec.