  In percutaneous intervention for treatment of coronary plaques, guidewire
navigation is a primary procedure for stent delivery. Steering a flexible
guidewire within coronary arteries requires considerable training, and the
non-linearity between the control operation and the movement of the guidewire
makes precise manipulation difficult. Here, we introduce a deep reinforcement
learning(RL) framework for autonomous guidewire navigation in a robot-assisted
coronary intervention. Using Rainbow, a segment-wise learning approach is
applied to determine how best to accelerate training using human demonstrations
with deep Q-learning from demonstrations (DQfD), transfer learning, and weight
initialization. `State' for RL is customized as a focus window near the
guidewire tip, and subgoals are placed to mitigate a sparse reward problem. The
RL agent improves performance, eventually enabling the guidewire to reach all
valid targets in `stable' phase. Our framework opens anew direction in the
automation of robot-assisted intervention, providing guidance on RL in physical
spaces involving mechanical fatigue.

Robotics;Computer Vision and Pattern Recognition