Multimodal Perception and Reinforcement Learning-Driven Intra-operative Real-time Surgical Strategy Support System for Mitral Valve Repair

P0142 

Abstract Submission 

Yu Han (1), Jiaxi Zhu (2), Haozhe Wang (3), Yanzai Zhou (4), Qiang Zhao (5), Xiaofeng Ye (6)

(1) Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, NA, (2) Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, (3) Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China, Shanghai, NA, (4) Ruijin Hospitai, Shanghai Jiao Tong University School of Medicine, Shanghai, China, Shanghai, NA, (5) Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, Shanghai, (6) Shanghai Jiaotong University School of Medcine, SHANGHAI, Shanghai

Yu Han    -  Contact Me
Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine

Jiaxi Zhu    -  Contact Me
Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine

Haozhe Wang    -  Contact Me
Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China

Yanzai Zhou    -  Contact Me
Ruijin Hospitai, Shanghai Jiao Tong University School of Medicine, Shanghai, China

*Qiang Zhao    -  Contact Me
Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine

Xiaofeng Ye    -  Contact Me
Shanghai Jiaotong University School of Medcine

Yu Han    -  Contact Me
Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine

Objective: Artificial intelligence may have significant potential to improve quality of mitral valve repair, optimize surgical training, and ultimately enable autonomous robotic surgery. This study aims to develop an artificial intelligence-powered intra-operative strategy support system for mitral valve repair.
Methods: All patients undergoing degenerative mitral valve repair with intraoperative video and preoperative echocardiogram images from January 2020 to December 2024 in a single center was enrolled in this study. The dataset was divided into training set, validation set and test set at an 8:1:1 ratio. This system learned to map patients' multimodal data (composed of intraoperative video, preoperative echocardiogram images and baseline information) to anatomy identification, pathology assessment, surgical technique planning and field demarcation. A self-supervised machine learning approach and a hybrid Convolutional Neural Network-Swin dual-stream network were utilized as the core deep learning architecture. For discrepancy analysis, the model-generated results were compared to that from expert surgeons as the gold standard.
Results: The study enrolled 305 patients, who were assigned to training set (n=241), validation set (n=31), and test set (n=33). Anatomy identification achieved robust performance (anterior leaflet: Dice scores=90.2%; posterior leaflet: Dice scores=89.1%; annulus: Dice scores=83.6%; chordae: Dice scores=87.3%; papillary muscles: Dice scores=85.4%). Pathology assessment demonstrated competent classification accuracy (anterior leaflet: exact accuracy=82.4%; posterior leaflet: exact accuracy=80.1%; annulus: exact accuracy=78.9%; and chordae: exact accuracy=82.2%). Surgical technique planning showed significant concordance, with 93.9% of the test set (31/33) being judged practicable with expert surgeons (AUC=0.78; Cohen's κ=0.862, p<0.001). Surgical field demarcation attained adequate performance with Dice score of 81.3%.
Conclusions: This preliminary artificial intelligence-driven system enabled precise anatomy identification, pathology assessment, surgical technique planning and field demarcation for mitral valve repair.

Mitral and Tricuspid Valve