Ambiguity-aware multimodal planning for collaborative robotics surgery

项目来源

日本学术振兴会基金(JSPS)

项目主持人

DAVILA Ana

项目受资助机构

名古屋大学

项目编号

25K21247

立项年度

2025

立项时间

未公开

项目级别

国家级

研究期限

未知 / 未知

受资助金额

4810000.00日元

学科

ヒューマンインタフェースおよびインタラクション関連

学科代码

未公开

基金类别

若手研究

关键词

Intelligent robotics ; Multimodal systems

参与者

未公开

参与机构

名古屋大学,未来社会創造機構

项目标书摘要:Outline of Research at the Start:This project addresses surgical robot adaptation through ambiguity-aware multimodal planning,by creating calibration datasets with ambiguous instructions,implementing context-sensitive vision-language planners,and designing clarification mechanisms to enhance healthcare safety。

  • 排序方式:
  • 1
  • /
  • 1.Adaptive Transfer Learning for Surgical Tool Presence Detection in Laparoscopic Videos Through Gradual Freezing Fine-Tuning

    • 关键词:
    • Advanced Analytics;Classification (of information);Deep learning;Image classification;Image enhancement;Laparoscopy;Learning systems;Medical image processing;Surgical equipment;Transfer learning ;Transplantation (surgical);Tuning;Advanced analysis;Fine tuning;Gradual freezing;Medical image analysis;Minimally-invasive surgery;Presence detections;Surgical tool classification;Surgical tools;Tool detection;Transfer learning
    • Davila, Ana;Colan, Jacinto;Hasegawa, Yasuhisa
    • 《International Journal of Imaging Systems and Technology》
    • 2025年
    • 35卷
    • 6期
    • 期刊

    Minimally invasive surgery can benefit significantly from automated surgical tool detection, enabling advanced analysis and assistance. However, the limited availability of annotated data in surgical settings poses a challenge for training robust deep learning models. This paper introduces a novel staged adaptive fine-tuning approach consisting of two steps: a linear probing stage to condition additional classification layers on a pre-trained CNN-based architecture and a gradual freezing stage to dynamically reduce the fine-tunable layers, aiming to regulate adaptation to the surgical domain. This strategy reduces network complexity and improves efficiency, requiring only a single training loop and eliminating the need for multiple iterations. We validated our method on the Cholec80 dataset, employing CNN architectures (ResNet-50 and DenseNet-121) pre-trained on ImageNet for detecting surgical tools in cholecystectomy endoscopic videos. Our results demonstrate that our method improves detection performance compared to existing approaches and established fine-tuning techniques, achieving a mean average precision (mAP) of 96.4%. To assess its broader applicability, the generalizability of the fine-tuning strategy was further confirmed on the CATARACTS dataset, a distinct domain of minimally invasive ophthalmic surgery. These findings suggest that gradual freezing fine-tuning is a promising technique for improving tool presence detection in diverse surgical procedures and may have broader applications in general image classification tasks. © 2025 Wiley Periodicals LLC.

    ...
  • 2.Bio-Inspired Fine-Tuning for Selective Transfer Learning in Image Classification

    • 关键词:
    • Image classification; adaptive transfer learning; adaptive transferlearning; fine-tuning; fine-tuning; evolutionary exploration;evolutionary exploration; bio-inspired optimization; bio-inspiredoptimization; medical imaging; medical imaging;CONVOLUTIONAL NEURAL-NETWORKS; INVERSE KINEMATICS
    • Davila, Ana;Colan, Jacinto;Hasegawa, Yasuhisa
    • 《IEEE ACCESS》
    • 2025年
    • 13卷
    • 期刊

    Deep learning has significantly advanced image analysis across diverse domains but often depends on large, annotated datasets for success. Transfer learning addresses this challenge by utilizing pre-trained models to tackle new tasks with limited labeled data. However, discrepancies between source and target domains can hinder effective transfer learning. We introduce BioTune, a novel adaptive fine-tuning technique utilizing evolutionary optimization. BioTune enhances transfer learning by optimally choosing which layers to freeze and adjusting learning rates for unfrozen layers. Through extensive evaluation on nine image classification datasets, spanning natural and specialized domains such as medical imaging, BioTune demonstrates superior accuracy and efficiency over state-of-the-art fine-tuning methods, including AutoRGN and LoRA, highlighting its adaptability to various data characteristics and distribution changes. Additionally, BioTune consistently achieves top performance across four different CNN architectures, underscoring its flexibility. Ablation studies provide valuable insights into the impact of BioTune's key components on overall performance.

    ...
  • 排序方式:
  • 1
  • /