Learning Expert Strategy for Autonomous Robotic Endovascular Intervention via Decoupled Procedural Execution

Yanxi Chen^1,2, Tianliang Yao³, Shaolong Tang¹, Jiyuan Zhao¹, Hengyu Hu^1,4,
Zhaoxing Li¹, Antonio Sánchez Egea⁵, Peng Qi^1,∗ This work has been accepted by IEEE/RSJ IROS 2026. Copyright may be transferred without notice, after which this version may no longer be accessible.This work is supported by the National Key Research and Development Program of China under Grant No. 2023YFB4705200, and the National Natural Science Foundation of China under Grant No. 52575034. The authors would like to thank Mr. Tao Liu from Shanghai Operation Robot Co., Ltd., for providing technical support in experiments. (*Corresponding Author: Peng Qi, email: pqi@tongji.edu.cn).¹Department of Control Science and Engineering, College of Electronics and Information Engineering, and Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 200092, China;²School of Mechanical Engineering, Tongji University, Shanghai 200092, China;³Department of Electronic Engineering, Faculty of Engineering, The Chinese University of Hong Kong, Hong Kong SAR 999077, China;⁴School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China;⁵Department of Mechanical Engineering, Universitat Politècnica de Catalunya (UPC), Barcelona 08034, Spain.

Abstract

Endovascular interventions are high-stakes procedures requiring precise device operation within complex and tortuous vascular anatomies. Autonomous endovascular navigation has the potential to standardize procedural quality and reduce the performance variability inherent in manual operation. Although Reinforcement Learning (RL) approaches have demonstrated promise in enabling autonomy in endovascular intervention, they often struggle with explicit constraint satisfaction and safety guarantees. To address these challenges, a learning-based expert strategy is introduced, enhancing procedural consistency in autonomous endovascular intervention by explicitly decoupling high-level strategic decision-making from low-level procedural execution. The proposed framework replicates the expert clinical decision-making process: a strategic RL policy generates global navigation intents, which are subsequently refined through an expert-informed execution module. This module ensures that robot movements strictly adhere to expert operational norms, real-time kinematic limits, and vessel safety constraints. Experimental evaluation across high-fidelity 3D simulations and a real-world robotic platform demonstrates that the proposed framework not only outperforms baseline policies but also effectively replicates expert-level proficiency. The framework achieves a high navigation success rate ( $>96\%$ ) and a 29.3% reduction in operational steps, which translates to enhanced operative efficiency and minimized device-vessel interaction. Furthermore, a 13% reduction in trajectory variance indicates superior procedural standardization, aligning autonomous behavior with established clinical norms. These results underscore its potential to enhance the predictability, safety, and consistency of robotic endovascular interventions.

I Introduction

Endovascular interventions are essential minimally invasive procedures for treating cardiovascular and neurovascular diseases [6]. These procedures require the precise manipulation of flexible guidewires and catheters through tortuous vascular structures under fluoroscopic guidance [7, 14]. Meanwhile, the operator must interpret 2D fluoroscopy images in real time, anticipate guidewire deformation, and apply precise control to avoid vessel-wall contact [17]. Such demands impose a steep learning curve, prolonged procedure times, and sustained radiation exposure [15]. Moreover, the variability in operator skill levels introduces significant inconsistency in procedural outcomes, underscoring the clinical need for intelligent robotic systems that can standardize interventional performance [3].

Refer to caption — Figure 1: The robot-assisted endovascular interventions are the minimally invasive procedures that utilize interventional guidewires, a C-arm X-ray machine, and endovascular robotics. (a) Manual operation: clinicians directly operate the robotic system based on their clinical expertise and procedural experience. (b) Autonomous operation: an automated console generates automatic instructions based on real-time fluoroscopes to endovascular robotics, where clinicians supervise the procedure. The lower panel illustrates guidewire navigation within complex vessel geometries, where predicted motion steps are executed toward the target region.

Robot-assisted autonomous endovascular navigation has emerged as a promising approach to reduce operator workload and standardize procedural quality. Within the evolving landscape of intervention autonomy [15, 3], reinforcement learning (RL) has emerged as a particularly promising paradigm, demonstrating the capacity to acquire guidewire navigation policies directly from simulated vascular environments [19, 13], adapting to complex anatomical geometries and nonlinear device dynamics through high-dimensional visual observations [9, 16, 20, 14]. However, existing RL-based methods exhibit limitations in clinical viability. Pure learning-based policies are prone to locally optimal decisions that manifest as redundant maneuvers and excessive navigation steps [11]. More critically, the absence of explicit constraint enforcement renders these methods unable to guarantee compliant device behavior under the uncertain contact conditions characteristic of real endovascular procedures, leading to unpredictable and potentially risky outcomes.

Structured regulation through optimization-based frameworks provides a robust mechanism to ensure technical compliance and motion consistency. Model Predictive Control (MPC), in particular, enables physically feasible trajectory generation by embedding operational constraints within an online optimization process [5]. Nevertheless, purely model-based formulations often struggle to capture the anatomy-dependent and highly nonlinear behavior of flexible guidewires, especially under conditions of incomplete sensing or unmodeled dynamics. Recent developments in autonomous systems have sought to unify these paradigms by coupling learning-based decision-making with optimization-based constraint regulation [8]. These hybrid architectures have demonstrated significant gains in robustness in complex domains such as agile flight and autonomous driving, motivating the adaptation of similar integration strategies to endovascular interventions.

Building upon this foundation, a learning expert strategy is proposed for autonomous robotic endovascular intervention via a decoupled procedural execution framework. The design leverages an explicit separation of strategic navigation intent from execution refined by clinical expertise, ensuring that robot movements consistently satisfy safety constraints and expert operational skills. An RL policy captures global navigation strategies from visual observations, while a downstream predictive refinement module enforces kinematic and vessel boundary constraints through real-time optimization. This decoupling frees the learning process from simultaneously resolving long-horizon planning and hard constraint compliance, enabling more effective strategy discovery. At deployment, the refinement module filters policy outputs to produce smooth, constraint-compliant actions, mirroring interventionalists who delineate procedural intent from precise device manipulation.

The proposed framework is validated in a high-fidelity 3D vascular simulation environment and further deployed on a physical robotic endovascular platform. Experimental results demonstrate that the framework achieves reliable autonomous navigation with improved efficiency, reducing navigation steps and trajectory variance while maintaining a high success rate compared with baseline RL methods. These results demonstrate the effectiveness of decoupling learning-based strategy acquisition from model-based constraint enforcement, and demonstrate the potential of this paradigm for reliable autonomous endovascular intervention.

The main contributions of this work are as follows:

•

A constraint-decoupled RL framework for autonomous endovascular guidewire navigation that separates strategic intent acquisition from operational constraint enforcement, providing a clinically aligned formulation for autonomous robotic endovascular intervention.
•

An online predictive motion refinement module that explicitly incorporates guidewire kinematics, curvature constraints, and vessel centerline adherence to ensure real-time constraint compliance and trajectory smoothness.
•

Comprehensive validation in both simulation and real-world robotic experiments, demonstrating improved navigation efficiency and trajectory consistency.

II Methodology

II-A Problem Formulation

The autonomous endovascular navigation task is formulated as a sequential decision-making problem under boundary constraints within a continuous vascular domain $\mathcal{V}\subset\mathbb{R}^{3}$ . The vascular environment is characterized by the vessel boundary $\partial\mathcal{V}$ and the anatomical centerline $\mathcal{C}=\{\mathbf{p}_{i}\}_{i=1}^{N}$ . Since clinical procedures are typically performed under 2D fluoroscopic guidance, the system state $\mathbf{x}_{t}\in\mathcal{X}$ at time $t$ is defined by the observable projected guidewire tip configuration:

\mathbf{x}_{t}=[x_{t},y_{t},v_{x,t},v_{y,t},\theta_{t}]^{T},

(1)

where $(x_{t},y_{t})$ , $(v_{x,t},v_{y,t})$ , and $\theta_{t}$ denote the tip position, velocity, and orientation in the imaging plane, respectively. The control input $\mathbf{u}_{t}\in\mathcal{U}$ represents the actuation applied at the proximal end of the guidewire by the robotic manipulator:

\mathbf{u}_{t}=[v_{\text{insert},t},\omega_{t}]^{T},

(2)

where $v_{\text{insert},t}$ is the insertion velocity and $\omega_{t}$ is the rotational velocity.

The primary objective is to generate an optimal control strategy that drives the guidewire from an initial state $\mathbf{x}_{0}$ to a target anatomical location $\mathbf{x}_{\text{goal}}\in\mathcal{C}$ . This navigation process must minimize cumulative procedural costs while adhering to a set of operational constraints $\mathcal{S}_{\text{bound}}$ , which include maintaining vessel wall clearance, adhering to curvature limits $\kappa_{\max}$ to prevent device deformation, and tracking the vascular centerline. This task is modeled as a Constrained Markov Decision Process (CMDP) and is addressed through a constraint-decoupled framework that separates global strategic intent from local motion refinement.

II-B Integrated Strategy-Execution Architecture

The proposed system utilizes a dual-stage architecture designed to mirror the clinical decision-making paradigm of interventionalists. Rather than employing an end-to-end mapping, the framework separates the acquisition of strategic navigation intent from the enforcement of operational constraints. As illustrated in Fig. 2, the framework comprises three functional modules: a visual perception pipeline, a strategic navigation policy, and a predictive motion refinement module.

The process begins with the perception module, which extracts the observation vector $\mathbf{o}_{t}$ representing the guidewire state and environmental features from fluoroscopic imagery. These observations are provided to the strategic navigation policy $\pi_{\phi}$ , which is trained in a high-fidelity 3D vascular simulation to learn global navigation tactics. The policy generates a reference control action, or navigation intent $\mathbf{u}_{\text{RL}}\sim\pi_{\phi}(\cdot|\mathbf{o}_{t})$ , aimed at maximizing long-term task success and procedural efficiency. To ensure rigorous adherence to technical constraints during execution, $\mathbf{u}_{\text{RL}}$ serves as the reference input to the downstream predictive refinement module. This module solves a finite-horizon optimization problem to filter the control actions, utilizing a simplified kinematic model $f(\mathbf{x}_{k},\mathbf{u}_{k})$ to predict future trajectories while enforcing explicit constraints on boundary clearance and curvature. The resulting optimal action $\mathbf{u}_{0}^{*}$ is subsequently executed by the robotic manipulator. This decoupling leverages the strategic adaptability for complex path planning while maintaining the operational consistency and smoothness provided by optimization-based control.

II-C Visual Perception and State Estimation

Real-time state estimation is achieved through a vision processing pipeline that interprets fluoroscopic feedback. The system acquires images via industrial cameras to track the interactions between the guidewire, catheter, and sheath. To localize the guidewire tip position $\mathbf{p}_{\text{tip}}=(x_{\text{tip}},y_{\text{tip}})$ , a YOLOv5 object detection model is employed to provide bounding box coordinates, from which the pixel-level center is derived. To mitigate detection failures, a temporal consistency check utilizes the last known valid position when detection confidence falls below a set threshold.

High-dimensional visual features are extracted to provide context regarding the vascular structure. A ResNet50 convolutional neural network, with the final classification layer removed, processes the standardized input images to yield a 2048-dimensional feature vector $\mathbf{f}_{\text{img}}$ that implicitly encodes spatial relationships and anatomical textures. The composite observation vector $\mathbf{o}_{t}$ aggregates the explicit geometric state and implicit visual features:

\mathbf{o}_{t}=[\mathbf{p}_{\text{tip}},\mathbf{f}_{\text{img}},\mathbf{u}_{t-1},\mathbf{p}_{\text{target}}],

(3)

where $\mathbf{u}_{t-1}$ is the previous control action and $\mathbf{p}_{\text{target}}$ is the goal position. This multi-modal representation enables the policy to reason about both the immediate tool configuration and the broader anatomical context.

II-D Strategic Navigation Policy

The strategic navigation behavior is governed by a policy trained using Soft Actor-Critic (SAC), an off-policy algorithm that maximizes an entropy-regularized objective. The policy network $\pi_{\phi}(\mathbf{u}_{t}|\mathbf{o}_{t})$ outputs a distribution over continuous navigation intents, while the Q-function networks $Q_{\theta}(\mathbf{o}_{t},\mathbf{u}_{t})$ estimate the expected return.

The training process is conducted within a simulated 3D vascular environment modeling complex aortic arch anatomy. The reward function $R(\mathbf{x}_{t},\mathbf{u}_{t})$ is designed to balance procedural completion with efficiency and boundary awareness. The total reward at each timestep is defined as:

R=r_{\text{target}}+r_{\text{path}}+r_{\text{step}}+r_{\text{boundary}}.

(4)

The target reaching term $r_{\text{target}}$ assigns a sparse positive reward $R_{\text{target}}$ upon arrival at the goal. Path efficiency is encouraged via $r_{\text{path}}=w_{\text{path}}\Delta d_{\text{path}}$ , which rewards the reduction in geodesic distance along the centerline. A constant step penalty $r_{\text{step}}$ is applied to incentivize rapid completion. The boundary penalty $r_{\text{boundary}}$ imposes a negative cost whenever the guidewire tip violates the minimum safe distance $d_{\min}$ from the vessel wall. This penalty serves as a behavioral bias during training, guiding the policy toward the centerline while the downstream refinement module ensures strict constraint compliance during deployment.

II-E Predictive Motion Refinement

To ensure technical compliance and kinematic feasibility, the reference intents generated by the policy, denoted as $\mathbf{u}_{\text{RL}}$ , are then refined by a Nonlinear MPC layer. The module solves a constrained optimization problem over a finite prediction horizon $H$ .

II-E1 Dynamics Model

The prediction module employs a discrete-time formulation of guidewire kinematics. The state evolution $f(\mathbf{x}_{k},\mathbf{u}_{k})$ is defined as:

$\displaystyle x_{k+1}$	$\displaystyle=x_{k}+v_{x,k}\Delta t,$	(5)
$\displaystyle y_{k+1}$	$\displaystyle=y_{k}+v_{y,k}\Delta t,$	(6)
$\displaystyle v_{x,k+1}$	$\displaystyle=\alpha v_{\text{insert},k}\cos(\theta_{k})+(1-\alpha)v_{x,k},$	(7)
$\displaystyle v_{y,k+1}$	$\displaystyle=\alpha v_{\text{insert},k}\sin(\theta_{k})+(1-\alpha)v_{y,k},$	(8)
$\displaystyle\theta_{k+1}$	$\displaystyle=\theta_{k}+\omega_{k}\Delta t,$	(9)

where $\Delta t$ is the sampling time and $\alpha\in(0,1)$ is a smoothing factor acting as a low-pass filter to model the inertial response of the physical device.

II-E2 Optimization Formulation

At each control step $t$ , the NMPC solves for the optimal sequence of control inputs $\mathbf{u}_{0:H-1}^{*}$ that minimizes the deviation from the strategic intent and the target trajectory. The optimization problem is formulated as:

\min_{\mathbf{u}_{0:H-1}}\sum_{k=0}^{H-1}(\|\mathbf{x}_{k}-\mathbf{x}_{\text{ref},k}\|^{2}_{\mathbf{Q}_{\text{ref}}}+\|\mathbf{u}_{k}-\mathbf{u}_{\text{RL}}\|^{2}_{\mathbf{R}})+J_{\text{terminal}}(\mathbf{x}_{H}),

(10)

where $\mathbf{x}_{\text{ref},k}$ is a target-directed reference trajectory, and $\mathbf{Q}_{\text{ref}}$ and $\mathbf{R}$ are weighting matrices. The term $\|\mathbf{u}_{k}-\mathbf{u}_{\text{RL}}\|^{2}_{\mathbf{R}}$ penalizes deviations from the learned strategic intent, ensuring the refinement module preserves the navigation tactics unless operational constraints require intervention. The optimization is subject to the following constraints for $k=0,\ldots,H-1$ :

$\displaystyle\mathbf{x}_{k+1}$	$\displaystyle=f(\mathbf{x}_{k},\mathbf{u}_{k}),$	(11)
$\displaystyle\mathbf{u}_{\min}$	$\displaystyle\leq\mathbf{u}_{k}\leq\mathbf{u}_{\max},$	(12)
$\displaystyle\frac{\|\omega_{k}\|}{\|v_{\text{insert},k}\|+\epsilon}$	$\displaystyle\leq\kappa_{\max},$	(13)
$\displaystyle d_{\mathcal{C}}(\mathbf{x}_{k})$	$\displaystyle\leq d_{\max},$	(14)

where $\mathbf{u}_{\min}$ and $\mathbf{u}_{\max}$ are actuator limits, and $\kappa_{\max}$ represents the curvature threshold. The constraint $d_{\mathcal{C}}(\mathbf{x}_{k})\leq d_{\max}$ enforces vessel boundary compliance. The optimization is implemented using the CasADi framework [1] with the IPOPT solver [2], applying a receding horizon strategy that executes only the first action $\mathbf{u}_{0}^{*}$ . This formulation effectively decouples strategic planning from real-time constraint enforcement.

III Experiments and Results

This section presents the evaluation of the proposed MPC-constrained RL framework for autonomous endovascular guidewire navigation in both simulation and physical settings. Systematic navigation tasks and ablation studies performed in a 3D vascular simulation environment validate the performance of this framework for endovascular procedures. Then, the trained policy was deployed on a robotic endovascular platform for real-world assessments.

III-A Experimental Configuration

The simulation platform is built upon the Sim4EndoR environment [13], which integrates SOFA v23.06 [18] with SofaPython3, BeamAdapter, and Blender. This modular setup supports the incorporation of patient-specific 3D vascular models and diverse robotic intervention scenarios. The guidewire is modeled using a beam-based finite element approach [12], featuring a J-shaped configuration with a straight body segment (0.89 mm diameter) and a reduced-diameter tip (0.7 mm diameter). The mechanical stiffness is specifically tuned to match commercially available guidewires to ensure realistic navigation dynamics within tortuous vasculature.

Training and simulation evaluations are executed on a workstation equipped with an Intel® Core™ i9-14900HX CPU (16 cores) and an NVIDIA GeForce RTX 4060 GPU. The software implementation utilizes PyTorch 1.11.0 and Python 3.8. The strategic navigation policy is developed using the Gymnasium library [10], while the predictive motion refinement module is implemented via the CasADi optimization framework.

III-B Strategy-Based Learning Configuration

III-B1 Algorithm and Hyperparameters

The strategic navigation policy is acquired through the Soft Actor-Critic (SAC) algorithm [4], selected for its off-policy efficiency and entropy-regularized exploration, which is critical for handling the highly nonlinear dynamics of endovascular navigation. Key hyperparameters, selected to ensure stable convergence, are detailed in Table I.

TABLE I: Hyperparameter Settings for Constraint-Decoupled Learning

Parameter	Value
Learning rate ( $lr$ )	$3.22\times 10^{-4}$
Discount factor ( $\gamma$ )	$0.99$
Target update rate ( $\tau$ )	$0.005$
Batch size	$32$
Replay buffer size	$10^{4}$ episodes
Reward scaling	$1.0$
LR end factor	$0.15$
LR decay steps	$6\times 10^{6}$
LSTM hidden nodes	$700$
MLP architecture	$3\times 400$
Entropy temperature ( $\alpha$ )	Auto-adjusted

III-B2 Network Architecture

The policy and twin Q-function networks are implemented as multi-layer perceptrons (MLPs) integrated with a shared Long Short-Term Memory (LSTM) embedder. This architecture captures temporal dependencies in the observation stream $\mathbf{o}_{t}$ . The shared LSTM processes raw observations into an embedded state $\mathbf{z}_{t}\in\mathbb{R}^{700}$ , which is then passed to the policy network comprising three hidden layers of 400 neurons each with ReLU activations. The output layer parameterizes a Gaussian distribution via $\bm{\mu}_{\phi}(\mathbf{z}_{t})$ and $\log\bm{\sigma}_{\phi}(\mathbf{z}_{t})$ . The twin Q-networks mirror this MLP structure, utilizing clipped double Q-learning to mitigate overestimation bias.

III-B3 Reward Hierarchy and Decoupling

The reward function is structured to balance procedural intent and navigation efficiency. The weighting terms follow the hierarchy $w_{\text{target}}\gg w_{\text{path}}\gg|w_{\text{step}}|$ , ensuring that task completion remains the primary learning signal while path efficiency and step minimization act as secondary shaping terms. Notably, in accordance with the constraint-decoupled paradigm, vessel-wall boundary constraints are not explicitly enforced within the reward function during the phase. This design allows the strategic policy to focus exclusively on acquiring high-level navigation tactics, delegating boundary compliance and technical regulation to the downstream refinement module.

III-C Simulation Evaluation

Quantitative evaluation was conducted in an endovascular intervention simulator to assess the efficacy of the proposed navigation paradigm. All experiments were performed over 50 navigation episodes with identical initialization conditions to ensure statistical consistency.

III-C1 Ablation on Refinement Weighting Factor

An ablation study was conducted by varying the normalized refinement weight $\lambda_{\text{MPC}}$ from $0.0$ to $1.0$ to identify the optimal balance between policy intent and constraint refinement. Across the interval $\lambda_{\text{MPC}}\in[0.0,0.5]$ , the navigation success rate consistently remained above 85%, indicating that moderate refinement authority does not compromise task completion. However, when $\lambda_{\text{MPC}}$ exceeded 0.5, performance deteriorated markedly, with the success rate decreasing from 70% to 40% as $\lambda_{\text{MPC}}$ increased further. This degradation suggests that excessive dominance of the optimization layer suppresses the learned strategic policy. In this regime, the control signal is primarily governed by local constraint satisfaction, effectively reducing the system to a tracking-oriented controller lacking sufficient global navigation guidance. This behavior aligns with the architectural principle of the proposed framework, where the RL policy generates high-level intent and the MPC module functions as a subordinate feasibility refinement layer rather than a primary decision-maker. Fig. 3(a) reports results within the representative range $[0,0.5]$ , where navigation performance exhibits a trade-off between policy autonomy and constraint authority. When $\lambda_{\text{MPC}}$ is small ( $<0.2$ ), the optimization layer provides limited corrective influence, resulting in behavior similar to the baseline RL policy, including inefficient oscillatory adjustments near bifurcations. Conversely, when $\lambda_{\text{MPC}}$ is large ( $>0.4$ ), the MPC layer dominates the control signal, which tends to suppress the learned navigation strategy and reduce adaptability to complex vessel geometries. An intermediate value of $\lambda_{\text{MPC}}=0.1$ achieved the highest success rate and optimal navigation efficiency; this value was therefore selected for all subsequent comparisons.

III-C2 Trajectory Consistency Analysis

Trajectory density was evaluated under stochastic rollouts from identical start and goal configurations to quantify the effect of the optimization layer on procedural repeatability. Superimposed 3D trajectories and probability density heatmaps in Fig. 3(b-g) illustrate that the integrated framework produces a noticeably more concentrated distribution compared to the diffuse patterns of the baseline. Quantitative metrics in Table II confirm this observation, with the lateral trajectory width in the $x$ – $y$ plane ( $\sigma_{\text{w,xy}}$ ) decreasing by $11.4\%$ . Additionally, the spatial spread along the primary navigation axis ( $\sigma_{x}$ ) was reduced by $13.0\%$ , indicating that the framework yields more consistent and predictable trajectories.

TABLE II: Comparison of trajectory consistency metrics.

Method	$\sigma_{x}$	$\sigma_{y}$	$\sigma_{z}$	$\sigma_{\text{w,xy}}$	$\sigma_{\text{w,xz}}$	$\sigma_{\text{w,3D}}$
Baseline RL	$16.86$	$11.77$	$82.50$	$6.50$	$13.98$	$17.59$
Ours	$14.67$	$11.02$	$78.87$	$5.76$	$13.11$	$17.11$
Reduction	$13.0\%$	—	$4.4\%$	$11.4\%$	$6.2\%$	$2.7\%$

Note: All $\sigma$ values are in mm. Lateral spreads $\sigma_{w,xy}$ and $\sigma_{w,xz}$ are derived via PCA (lower is better).

TABLE III: Comparison of Performance Evaluations in Endovascular Navigation Tasks Across Multiple Anatomical Geometries.

Metric	Brachiocephalic			Carotid			Subclavian
Metric	SAC	DDPG	Ours	SAC	DDPG	Ours	SAC	DDPG	Ours
Success Rate $\uparrow$	92% (46/50)	86% (43/50)	98% (49/50)	94% (47/50)	92% (46/50)	96% (48/50)	90% (45/50)	84% (42/50)	98% (49/50)
Steps $\downarrow$	85.89 $\pm$ 18.23	88.51 $\pm$ 9.89	71.55 $\pm$ 10.14	94.24 $\pm$ 14.65	118.24 $\pm$ 25.62	71.74 $\pm$ 3.62	98.75 $\pm$ 8.21	128.74 $\pm$ 13.21	70.92 $\pm$ 5.18
Path Ratio $\uparrow$	0.957 $\pm$ 0.075	0.978 $\pm$ 0.032	0.984 $\pm$ 0.008	0.965 $\pm$ 0.069	0.959 $\pm$ 0.019	0.981 $\pm$ 0.016	0.982 $\pm$ 0.007	0.985 $\pm$ 0.023	0.986 $\pm$ 0.005
Reward $\uparrow$	2.198 $\pm$ 0.195	2.205 $\pm$ 0.186	2.215 $\pm$ 0.056	2.123 $\pm$ 0.165	2.105 $\pm$ 0.086	2.142 $\pm$ 0.074	2.038 $\pm$ 0.112	2.045 $\pm$ 0.074	2.221 $\pm$ 0.064

III-C3 Comparative Study on Navigation Performance Across Multiple Anatomical Geometries.

Using the selected refinement weight, the proposed framework was compared against pure RL policies trained under identical settings. As summarized in Table III, the integrated framework consistently improves both navigation safety and efficiency across the Brachiocephalic, Carotid, and Subclavian anatomies. Aggregated across three geometries, the success rate is increased to above 96%, demonstrating enhanced robustness and stability. Meanwhile, the average procedural steps are reduced by 29.3% compared to baseline RL policies, with a statistically significant difference confirmed by Welch’s t-test (both $p<0.003$ ), indicating improved procedural efficiency. Furthermore, the higher path ratio and reward suggest closer adherence to the vessel centerline, reflecting safer and smoother trajectories. These results indicate that the proposed strategy not only increases task success rate but also improves procedural efficiency while maintaining high reliability.

III-D In vitro Validation on Vascular Phantoms

The proposed framework was deployed on the ALLVAS™ robotic system (Shanghai Operation Robot Co., Ltd., Shanghai, China), which integrates a precision manipulator with a dedicated guidewire manipulation mechanism. The vision system utilized a high-resolution camera (Hikvision, Hangzhou, China) for real-time feedback, employing YOLOv5 for the detection of the guidewire tip, catheter, and sheath, while ResNet50 was used for visual feature extraction. In this deployment, the RL policy generated high-level navigation intent based on visual observations, which was then refined by a CasADi-based MPC controller. The MPC layer enforced constraints including vessel-boundary clearance thresholds, curvature limits, and velocity bounds tailored to the physical properties of the guidewire.

Validation was conducted through 20 navigation trials within a silicone vascular phantom maintained at 36.8 ^∘C by a circulation system to simulate physiological conditions (Fig. 4). The framework achieved a $90.0\%$ success rate (18 out of 20 trials), with an average navigation time of $4.2\pm 0.8$ minutes and $156\pm 32$ control steps per trial. Although a moderate performance drop was observed compared to simulation results, this is consistent with the expected sim-to-real gap caused by perception uncertainty. Notably, zero vessel-boundary violations were recorded across all physical experiments, as the MPC layer successfully intervened during potential buckling or near-boundary events.

IV Discussion

This study proposes an integrated navigation framework that combines learning-driven strategy acquisition with predictive, optimization-based control to address the trade-off of efficiency and precision in autonomous endovascular interventions, specifically the requirement for precise constraint satisfaction within confined and tortuous vascular anatomies. While conventional learning-based methods offer high strategic adaptability, they often struggle to maintain strict boundary adherence and exhibit unstable oscillatory behaviors near sharp bifurcations. Experimental results demonstrate that the proposed optimization-based refinement layer effectively resolves these limitations by filtering high-level policy commands through kinematic and boundary constraints. This synergy ensures superior trajectory consistency and motion smoothness, providing a robust mechanism to maintain predictable device behavior within delicate anatomical spaces.

The architecture mirrors the multi-stage decision logic employed by interventionalists who separate global procedural planning from the nuanced manipulation of instruments. By replicating this workflow, the proposed strategy provides an expert-informed mechanism for maintaining consistent vessel boundary clearance without compromising navigation goals. Furthermore, the modular design allows optimization constraints to be adjusted for diverse device configurations or patient-specific anatomies without necessitating policy retraining, establishing a versatile foundation for autonomous interventional robotics.

The transition from simulation to the physical platform resulted in a reduction in success rate from 98.0% to 90.0%, which may be attributable to unmodeled physical factors such as non-linear friction, mechanical hysteresis, and visual occlusions. Nevertheless, the persistent zero-violation rate in physical trials demonstrates that the optimization layer effectively mitigates policy-level deviations, maintaining procedural safety in real-world settings. Future efforts will prioritize the integration of high-fidelity biomechanical models, including vessel wall compliance and complex guidewire deformation, to further bridge the sim-to-real gap. Subsequent in vivo studies will evaluate the framework’s clinical efficacy and robustness under dynamic conditions.

V Conclusion

This study introduces a learning expert strategy designed to optimize the consistency and efficiency of autonomous endovascular navigation. By decoupling strategic decision-making from procedural execution, the system ensures strict adherence to expert-aligned procedural standards and vessel safety constraints while achieving superior operational smoothness. This architecture inherently replicates the expert decision-making process, where interventionalists distinguish high-level procedural intent from fine-grained device manipulation, thereby providing a robust mechanism to mitigate intraoperative risks. Such alignment with established interventional workflows underscores the potential of the proposed method for enhancing the precision and standardization of robotic interventions.

The modular structure of this framework enables its adaptation to various vascular geometries and device configurations, providing a scalable foundation for autonomous intervention robotics. To further refine this approach, future research will integrate advanced biomechanical models, including vessel wall compliance and non-linear guidewire deformation, to enhance the simulation-to-reality transition. Furthermore, in vivo animal trials need to be conducted to evaluate the clinical performance and translation potential within complex, dynamic physiological environments.

References

[1] J. A. E. Andersson, J. Gillis, G. Horn, J. B. Rawlings, and M. Diehl (2019) CasADi – A software framework for nonlinear optimization and optimal control. Mathematical Programming Computation 11 (1), pp. 1–36. External Links: Document Cited by: §II-E2.
[2] N. Chiang and V. M. Zavala (2016) An inertia-free filter line-search algorithm for large-scale nonlinear programming. Computational Optimization and Applications 64 (2), pp. 327–354. Cited by: §II-E2.
[3] P. E. Dupont and A. Degirmenci (2025) The grand challenges of learning medical robot autonomy. Science Robotics 10 (104), pp. eadz8279. Cited by: §I, §I.
[4] T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, et al. (2018) Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905. Cited by: §III-B1.
[5] G. Li and G. Loianno (2023) Nonlinear model predictive control for cooperative transportation and manipulation of cable suspended payloads with multiple quadrotors. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5034–5041. Cited by: §I.
[6] S. Liang, Z. Liu, G. Liu, T. Yao, C. Yang, and P. Qi (2026) Self-supervised x-ray coronary angiography segmentation with vessel-aware synthesis learning. IEEE Journal of Biomedical and Health Informatics, pp. 1–11. Cited by: §I.
[7] A. Pore, Z. Li, D. Dall’Alba, A. Hernansanz, E. De Momi, A. Menciassi, A. C. Gelpi, J. Dankelman, P. Fiorini, and E. Vander Poorten (2023) Autonomous navigation for robot-assisted intraluminal and endovascular procedures: a systematic review. IEEE Transactions on Robotics 39 (4), pp. 2529–2548. Cited by: §I.
[8] A. Romero, E. Aljalbout, Y. Song, and D. Scaramuzza (2025) Actor–critic model predictive control: differentiable optimization meets reinforcement learning for agile flight. IEEE Transactions on Robotics 42, pp. 673–692. Cited by: §I.
[9] V. Scarponi, F. Lecomte, M. Duprez, F. Nageotte, and S. Cotin (2024) Autonomous guidewire navigation in dynamic environments. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6931–6938. Cited by: §I.
[10] M. Towers, A. Kwiatkowski, J. Terry, J. U. Balis, G. De Cola, T. Deleu, M. Goulão, A. Kallinteris, M. Krimmel, A. KG, et al. (2024) Gymnasium: a standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032. Cited by: §III-A.
[11] H. Wang, X. Tan, T. Yao, Z. Fang, P. Qi, and X. Qiu (2025) Learning expressive task embeddings and sample-efficient exploration for context shift reduction in offline meta-reinforcement environment. IEEE Transactions on Cognitive and Developmental Systems (), pp. 1–15. Cited by: §I.
[12] Y. Wei, S. Cotin, J. Dequidt, C. Duriez, J. Allard, and E. Kerrien (2012-08) A (Near) Real-Time Simulation Method of Aneurysm Coil Embolization. In Aneurysm, Y. Murai (Ed.), pp. 223–248. External Links: Link, Document Cited by: §III-A.
[13] T. Yao, M. Ban, B. Lu, Z. Pei, and P. Qi (2025) Sim4EndoR: a reinforcement learning centered simulation platform for task automation of endovascular robotics. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 824–830. Cited by: §I, §III-A.
[14] T. Yao, B. Li, B. Lu, Z. Pei, Y. Yuan, and P. Qi (2025) Real-time 3D guidewire reconstruction from intraoperative DSA images for robot-assisted endovascular interventions. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 17344–17351. Cited by: §I, §I.
[15] T. Yao, B. Lu, M. Kowarschik, Y. Yuan, H. Zhao, S. Ourselin, K. Althoefer, J. Ge, and P. Qi (2026) Advancing embodied intelligence in robotic-assisted endovascular procedures: a systematic review of AI solutions. IEEE Reviews in Biomedical Engineering 19 (), pp. 248–266. Cited by: §I, §I.
[16] T. Yao, Z. Pei, Y. Li, Y. Yuan, and P. Qi (2025) Real-time guidewire tip tracking using a siamese network for image-guided endovascular procedures. Advanced Intelligent Systems 7 (10), pp. 2500425. Cited by: §I.
[17] T. Yao, C. Wang, X. Wang, X. Li, Z. Jiang, and P. Qi (2023) Enhancing percutaneous coronary intervention with heuristic path planning and deep-learning-based vascular segmentation. Computers in Biology and Medicine 166, pp. 107540. Cited by: §I.
[18] T. Yao, H. Wang, B. Lu, J. Ge, Z. Pei, M. Kowarschik, L. Sun, L. Seneviratne, and P. Qi (2025) Sim2real learning with domain randomization for autonomous guidewire navigation in robotic-assisted endovascular procedures. IEEE Transactions on Automation Science and Engineering 22 (), pp. 13842–13854. Cited by: §III-A.
[19] T. Yao, Y. Xu, H. Wang, X. Qiu, K. Althoefer, and P. Qi (2025) Multi-agent fuzzy reinforcement learning with llm for cooperative navigation of endovascular robotics. IEEE Transactions on Fuzzy Systems (), pp. 1–11. Cited by: §I.
[20] J. Zhao, Z. Shi, W. Tian, T. Yao, D. Liu, T. Liu, Y. Wu, and P. Qi (2026) Vision-based reasoning with topology-encoded graphs for anatomical path disambiguation in robot-assisted endovascular navigation. In 2026 IEEE International Conference on Robotics and Automation (ICRA), pp. 21617–21624. Cited by: §I.