PS53. Automatic Detection of the Pulmonary Artery During Robotic Right Lower Lobectomy Using Multi-Headed Deep Learning
Presented During: Thoracic Poster Viewing Showcase
Harvard Medical School
Boston, MA
United States - Contact Me
Arian Mansur is a second-year medical student at Harvard Medical School. He received his bachelor’s degree in Human Developmental and Regenerative Biology with a secondary in Global Health and Health Policy from Harvard University, where he graduated Magna cum laude with Highest Honors in the field. Arian’s research focus are on the treatments for early-stage small-cell lung cancer and on using advanced statistical modeling to address important questions in thoracic surgery.
Saturday, May 6, 2023: 8:00 AM - Tuesday, May 9, 2023: 11:45 AM
Los Angeles Convention Center
Room: Outside of Room 408
Description
Objective: To develop a computer vision-based approach to reliably identify the pulmonary artery during robotic right lower lobectomy.
Methods: Four patients with biopsy-proven Stage I non-small-cell lung cancer (NSCLC) of the right lower lobe underwent robotic lobectomy with mediastinal lymph node dissection. Complete videos of each operation were obtained, and video fragments of the pulmonary artery were identified by two board-certified thoracic surgeons. Annotation masks of the pulmonary artery were then created using Computer Vision Annotation Tool (CVAT), an open-source web-based annotation tool. The labeled data was next used to train a state-of-the-art instance segmentation algorithm (Figure 1) called Mask Regional-Convolutional Neural Network (Mask R-CNN) using an 80:20 random sample split from three cases for training and validation, respectively. A fourth case was utilized for generalization testing. Three custom performance metrics commonly utilized in deep learning-based instance segmentation were used to evaluate our approach: Intersection over Union (IoU), Average Precision at IoU threshold of 50% (AP50) and at 75% (AP75).
Results: Annotation masks of the pulmonary artery were created in 1,883 images across four cases of robotic right lower lobectomy. 1,312 and 310 annotated images across three (75%) cases were used for training and validation, respectively; and 261 annotated images from one (25%) case were used for generalization testing. A mean IoU of 94.2% was achieved in the validation dataset. The AP50 attained by our model was 96.0% and AP75 was 89.1%. In generalizability testing, when the model was tested on data never exposed to it, it was able to partially generalize with a mean IoU of 22.0%.
Conclusions: Our study shows that our Mask R-CNN model was able to identify surgical anatomy during robotic lobectomy with high accuracy in the validation dataset. While still at an early stage, more variable labeled data and collaborative efforts are needed to improve the generalizability of our model. We envision such a computer vision system to be valuable in resident and early surgeon training. Future real-time augmentation of thoracic surgery video feedback can potentially be utilized to prevent major complications and improve surgical outcomes.
Methods: Four patients with biopsy-proven Stage I non-small-cell lung cancer (NSCLC) of the right lower lobe underwent robotic lobectomy with mediastinal lymph node dissection. Complete videos of each operation were obtained, and video fragments of the pulmonary artery were identified by two board-certified thoracic surgeons. Annotation masks of the pulmonary artery were then created using Computer Vision Annotation Tool (CVAT), an open-source web-based annotation tool. The labeled data was next used to train a state-of-the-art instance segmentation algorithm (Figure 1) called Mask Regional-Convolutional Neural Network (Mask R-CNN) using an 80:20 random sample split from three cases for training and validation, respectively. A fourth case was utilized for generalization testing. Three custom performance metrics commonly utilized in deep learning-based instance segmentation were used to evaluate our approach: Intersection over Union (IoU), Average Precision at IoU threshold of 50% (AP50) and at 75% (AP75).
Results: Annotation masks of the pulmonary artery were created in 1,883 images across four cases of robotic right lower lobectomy. 1,312 and 310 annotated images across three (75%) cases were used for training and validation, respectively; and 261 annotated images from one (25%) case were used for generalization testing. A mean IoU of 94.2% was achieved in the validation dataset. The AP50 attained by our model was 96.0% and AP75 was 89.1%. In generalizability testing, when the model was tested on data never exposed to it, it was able to partially generalize with a mean IoU of 22.0%.
Conclusions: Our study shows that our Mask R-CNN model was able to identify surgical anatomy during robotic lobectomy with high accuracy in the validation dataset. While still at an early stage, more variable labeled data and collaborative efforts are needed to improve the generalizability of our model. We envision such a computer vision system to be valuable in resident and early surgeon training. Future real-time augmentation of thoracic surgery video feedback can potentially be utilized to prevent major complications and improve surgical outcomes.
Presentation Duration
There is no formal oral presentation associated with this electronic poster. Your poster will be available for viewing at the poster kiosk located outside of the specialty room as well as in the Exhibit Hall, for the duration of the meeting.