  Robot-assisted intervention has shown reduced radiation exposure to
physicians and improved precision in clinical trials. However, existing
vascular robotic systems follow master-slave control mode and entirely rely on
manual commands. This paper proposes a novel offline reinforcement learning
algorithm, Conservative Actor-critic with SmOoth Gradient (CASOG), to learn
manipulation skills from human demonstrations on vascular robotic systems. The
proposed algorithm conservatively estimates Q-function and smooths gradients of
convolution layers to deal with distribution shift and overfitting issues.
Furthermore, to focus on complex manipulations, transitions with larger
temporal-difference error are sampled with higher probability. Comparative
experiments in a pre-clinical environment demonstrate that CASOG can deliver
guidewire to the target at a success rate of 94.00\% and mean backward steps of
14.07, performing closer to humans and better than prior offline reinforcement
learning methods. These results indicate that the proposed algorithm is
promising to improve the autonomy of vascular robotic systems.

Robotics