  As critically ill patients frequently develop anemia or coagulopathy,
transfusion of blood products is a frequent intervention in the Intensive Care
Units (ICU). However, inappropriate transfusion decisions made by physicians
are often associated with increased risk of complications and higher hospital
costs. In this work, we aim to develop a decision support tool that uses
available patient information for transfusion decision-making on three common
blood products (red blood cells, platelets, and fresh frozen plasma). To this
end, we adopt an off-policy batch reinforcement learning (RL) algorithm,
namely, discretized Batch Constrained Q-learning, to determine the best action
(transfusion or not) given observed patient trajectories. Simultaneously, we
consider different state representation approaches and reward design mechanisms
to evaluate their impacts on policy learning. Experiments are conducted on two
real-world critical care datasets: the MIMIC-III and the UCSF. Results
demonstrate that policy recommendations on transfusion achieved comparable
matching against true hospital policies via accuracy and weighted importance
sampling evaluations on the MIMIC-III dataset. Furthermore, a combination of
transfer learning (TL) and RL on the data-scarce UCSF dataset can provide up to
$17.02% improvement in terms of accuracy, and up to 18.94% and 21.63%
improvement in jump-start and asymptotic performance in terms of weighted
importance sampling averaged over three transfusion tasks. Finally, simulations
on transfusion decisions suggest that the transferred RL policy could reduce
patients' estimated 28-day mortality rate by 2.74% and decreased acuity rate by
1.18% on the UCSF dataset.

Machine Learning;Artificial Intelligence