Automatic Detection of the Pulmonary Artery During Robotic Right Lower Lobectomy Using Multi-Headed Deep Learning
Presented During:
Saturday, May 6, 2023: 8:00AM - Tuesday, May 9, 2023: 11:45AM
Los Angeles Convention Center
Posted Room Name:
Outside of Room 408
Abstract No:
PS053
Submission Type:
Abstract Submission
Authors:
Arian Mansur (1), Syed Abdul Khader (2), Vyaskh R K (2), Mika Jain (1), Pratik Chaudhari (3), Chi-Fu Jeffrey Yang (4), Sandeep Manjanna (2), Lana Schumacher (4)
Institutions:
(1) Harvard Medical School, Boston, MA, (2) Plaksha University, Mohali, NA, (3) University of Pennsylvania, Philadelphia, PA, (4) Massachusetts General Hospital, Boston, MA
Submitting Author:
Arian Mansur
-
Contact Me
Harvard Medical School
Harvard Medical School
Co-Author(s):
Syed Abdul Khader
-
Contact Me
Plaksha University
Plaksha University
Vyaskh R K
-
Contact Me
Plaksha University
Plaksha University
Mika Jain
-
Contact Me
Harvard Medical School
Harvard Medical School
Pratik Chaudhari
-
Contact Me
University of Pennsylvania
University of Pennsylvania
*Chi-Fu Yang
-
Contact Me
Massachusetts General Hospital
Massachusetts General Hospital
Sandeep Manjanna
-
Contact Me
Plaksha University
Plaksha University
*Lana Schumacher
-
Contact Me
Massachusetts General Hospital
Massachusetts General Hospital
Presenting Author:
Arian Mansur
-
Contact Me
N/A
N/A
Abstract:
Objective: To develop a computer vision-based approach to reliably identify the pulmonary artery during robotic right lower lobectomy.
Methods: Four patients with biopsy-proven Stage I non-small-cell lung cancer (NSCLC) of the right lower lobe underwent robotic lobectomy with mediastinal lymph node dissection. Complete videos of each operation were obtained, and video fragments of the pulmonary artery were identified by two board-certified thoracic surgeons. Annotation masks of the pulmonary artery were then created using Computer Vision Annotation Tool (CVAT), an open-source web-based annotation tool. The labeled data was next used to train a state-of-the-art instance segmentation algorithm (Figure 1) called Mask Regional-Convolutional Neural Network (Mask R-CNN) using an 80:20 random sample split from three cases for training and validation, respectively. A fourth case was utilized for generalization testing. Three custom performance metrics commonly utilized in deep learning-based instance segmentation were used to evaluate our approach: Intersection over Union (IoU), Average Precision at IoU threshold of 50% (AP50) and at 75% (AP75).
Results: Annotation masks of the pulmonary artery were created in 1,883 images across four cases of robotic right lower lobectomy. 1,312 and 310 annotated images across three (75%) cases were used for training and validation, respectively; and 261 annotated images from one (25%) case were used for generalization testing. A mean IoU of 94.2% was achieved in the validation dataset. The AP50 attained by our model was 96.0% and AP75 was 89.1%. In generalizability testing, when the model was tested on data never exposed to it, it was able to partially generalize with a mean IoU of 22.0%.
Conclusions: Our study shows that our Mask R-CNN model was able to identify surgical anatomy during robotic lobectomy with high accuracy in the validation dataset. While still at an early stage, more variable labeled data and collaborative efforts are needed to improve the generalizability of our model. We envision such a computer vision system to be valuable in resident and early surgeon training. Future real-time augmentation of thoracic surgery video feedback can potentially be utilized to prevent major complications and improve surgical outcomes.
Methods: Four patients with biopsy-proven Stage I non-small-cell lung cancer (NSCLC) of the right lower lobe underwent robotic lobectomy with mediastinal lymph node dissection. Complete videos of each operation were obtained, and video fragments of the pulmonary artery were identified by two board-certified thoracic surgeons. Annotation masks of the pulmonary artery were then created using Computer Vision Annotation Tool (CVAT), an open-source web-based annotation tool. The labeled data was next used to train a state-of-the-art instance segmentation algorithm (Figure 1) called Mask Regional-Convolutional Neural Network (Mask R-CNN) using an 80:20 random sample split from three cases for training and validation, respectively. A fourth case was utilized for generalization testing. Three custom performance metrics commonly utilized in deep learning-based instance segmentation were used to evaluate our approach: Intersection over Union (IoU), Average Precision at IoU threshold of 50% (AP50) and at 75% (AP75).
Results: Annotation masks of the pulmonary artery were created in 1,883 images across four cases of robotic right lower lobectomy. 1,312 and 310 annotated images across three (75%) cases were used for training and validation, respectively; and 261 annotated images from one (25%) case were used for generalization testing. A mean IoU of 94.2% was achieved in the validation dataset. The AP50 attained by our model was 96.0% and AP75 was 89.1%. In generalizability testing, when the model was tested on data never exposed to it, it was able to partially generalize with a mean IoU of 22.0%.
Conclusions: Our study shows that our Mask R-CNN model was able to identify surgical anatomy during robotic lobectomy with high accuracy in the validation dataset. While still at an early stage, more variable labeled data and collaborative efforts are needed to improve the generalizability of our model. We envision such a computer vision system to be valuable in resident and early surgeon training. Future real-time augmentation of thoracic surgery video feedback can potentially be utilized to prevent major complications and improve surgical outcomes.
Categories:
Lung Cancer
Image or Table
