query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : I want to build a healthcare decision support system that will recommend treatment plans for patients. In order to make the prediction process transparent and interpretable, I believe I could use reinforcement learning. Given the patient’s past medical history, I will spawn many processes, in each process an artificial or digital twin for the patient is spawned to mimic the patient’s health trajectories. In each process, the patient will receive many treatments, prescriptions, surgeries, and will experience a rest and recovery period, and eventually some form of complications or disease will terminate the patient, and thus terminate the process as well. I want to model this entire environment dynamic using reinforcement learning, conditioned on the patient's past medical record. I will model each dynamic in the environment as some form of numeric optimization problems where the human body strives to optimize its ability to recover and detect possible illness. I need the entire framework to be effective in casting optimization problems, and efficiently solve numeric planning problems involving obstacles. I will use this modeling approach to determine the best course of treatment for the patient to maximize the patient’s life expectancy.

positive:
Although reinforcement learning (RL) has tremendous success in many fields,
applying RL to real-world settings such as healthcare is challenging when the
reward is hard to specify and no exploration is allowed. In this work, we focus
on recovering clinicians' rewards in treating patients. We incorporate the
what-if reasoning to explain the clinician's treatments based on their
potential future outcomes. We use generalized additive models (GAMs) - a class
of accurate, interpretable models - to recover the reward. In both simulation
and a real-world hospital dataset, we show our model outperforms baselines.
Finally, our model's explanations match several clinical guidelines when
treating patients while we found the commonly-used linear model often
contradicts them.

positive:
The rapid increase in the percentage of chronic disease patients along with
the recent pandemic pose immediate threats on healthcare expenditure and
elevate causes of death. This calls for transforming healthcare systems away
from one-on-one patient treatment into intelligent health systems, to improve
services, access and scalability, while reducing costs. Reinforcement Learning
(RL) has witnessed an intrinsic breakthrough in solving a variety of complex
problems for diverse applications and services. Thus, we conduct in this paper
a comprehensive survey of the recent models and techniques of RL that have been
developed/used for supporting Intelligent-healthcare (I-health) systems. This
paper can guide the readers to deeply understand the state-of-the-art regarding
the use of RL in the context of I-health. Specifically, we first present an
overview for the I-health systems challenges, architecture, and how RL can
benefit these systems. We then review the background and mathematical modeling
of different RL, Deep RL (DRL), and multi-agent RL models. After that, we
provide a deep literature review for the applications of RL in I-health
systems. In particular, three main areas have been tackled, i.e., edge
intelligence, smart core network, and dynamic treatment regimes. Finally, we
highlight emerging challenges and outline future research directions in driving
the future success of RL in I-health systems, which opens the door for
exploring some interesting and unsolved problems.

positive:
Dynamic treatment recommendation systems based on large-scale electronic
health records (EHRs) become a key to successfully improve practical clinical
outcomes. Prior relevant studies recommend treatments either use supervised
learning (e.g. matching the indicator signal which denotes doctor
prescriptions), or reinforcement learning (e.g. maximizing evaluation signal
which indicates cumulative reward from survival rates). However, none of these
studies have considered to combine the benefits of supervised learning and
reinforcement learning. In this paper, we propose Supervised Reinforcement
Learning with Recurrent Neural Network (SRL-RNN), which fuses them into a
synergistic learning framework. Specifically, SRL-RNN applies an off-policy
actor-critic framework to handle complex relations among multiple medications,
diseases and individual characteristics. The "actor" in the framework is
adjusted by both the indicator signal and evaluation signal to ensure effective
prescription and low mortality. RNN is further utilized to solve the
Partially-Observed Markov Decision Process (POMDP) problem due to the lack of
fully observed states in real world applications. Experiments on the publicly
real-world dataset, i.e., MIMIC-3, illustrate that our model can reduce the
estimated mortality, while providing promising accuracy in matching doctors'
prescriptions.

positive:
Clinical decision making regarding treatments based on personal
characteristics leads to effective health improvements. Machine learning (ML)
has been the primary concern of diagnosis support according to comprehensive
patient information. However, the remaining prominent issue is the development
of objective treatment processes in clinical situations. This study proposes a
novel framework to plan treatment processes in a data-driven manner. A key
point of the framework is the evaluation of the "actionability" for personal
health improvements by using a surrogate Bayesian model in addition to a
high-performance nonlinear ML model. We first evaluated the framework from the
viewpoint of its methodology using a synthetic dataset. Subsequently, the
framework was applied to an actual health checkup dataset comprising data from
3,132 participants, to improve systolic blood pressure values at the individual
level. We confirmed that the computed treatment processes are actionable and
consistent with clinical knowledge for lowering blood pressure. These results
demonstrate that our framework could contribute toward decision making in the
medical field, providing clinicians with deeper insights.

positive:
Off-policy reinforcement learning enables near-optimal policy from suboptimal
experience, thereby provisions opportunity for artificial intelligence
applications in healthcare. Previous works have mainly framed patient-clinician
interactions as Markov decision processes, while true physiological states are
not necessarily fully observable from clinical data. We capture this situation
with partially observable Markov decision process, in which an agent optimises
its actions in a belief represented as a distribution of patient states
inferred from individual history trajectories. A Gaussian mixture model is
fitted for the observed data. Moreover, we take into account the fact that
nuance in pharmaceutical dosage could presumably result in significantly
different effect by modelling a continuous policy through a Gaussian
approximator directly in the policy space, i.e. the actor. To address the
challenge of infinite number of possible belief states which renders exact
value iteration intractable, we evaluate and plan for only every encountered
belief, through heuristic search tree by tightly maintaining lower and upper
bounds of the true value of belief. We further resort to function
approximations to update value bounds estimation, i.e. the critic, so that the
tree search can be improved through more compact bounds at the fringe nodes
that will be back-propagated to the root. Both actor and critic parameters are
learned via gradient-based approaches. Our proposed policy trained from real
intensive care unit data is capable of dictating dosing on vasopressors and
intravenous fluids for sepsis patients that lead to the best patient outcomes.

positive:
Most of the existing medicine recommendation systems that are mainly based on
electronic medical records (EMRs) are significantly assisting doctors to make
better clinical decisions benefiting both patients and caregivers. Even though
the growth of EMRs is at a lighting fast speed in the era of big data, content
limitations in EMRs restrain the existed recommendation systems to reflect
relevant medical facts, such as drug-drug interactions. Many medical knowledge
graphs that contain drug-related information, such as DrugBank, may give hope
for the recommendation systems. However, the direct use of these knowledge
graphs in the systems suffers from robustness caused by the incompleteness of
the graphs. To address these challenges, we stand on recent advances in graph
embedding learning techniques and propose a novel framework, called Safe
Medicine Recommendation (SMR), in this paper. Specifically, SMR first
constructs a high-quality heterogeneous graph by bridging EMRs (MIMIC-III) and
medical knowledge graphs (ICD-9 ontology and DrugBank). Then, SMR jointly
embeds diseases, medicines, patients, and their corresponding relations into a
shared lower dimensional space. Finally, SMR uses the embeddings to decompose
the medicine recommendation into a link prediction process while considering
the patient's diagnoses and adverse drug reactions. To our best knowledge, SMR
is the first to learn embeddings of a patient-disease-medicine graph for
medicine recommendation in the world. Extensive experiments on real datasets
are conducted to evaluate the effectiveness of proposed framework.

negative:
In the modern healthcare system, rapidly expanding costs/complexity, the
growing myriad of treatment options, and exploding information streams that
often do not effectively reach the front lines hinder the ability to choose
optimal treatment decisions over time. The goal in this paper is to develop a
general purpose (non-disease-specific) computational/artificial intelligence
(AI) framework to address these challenges. This serves two potential
functions: 1) a simulation environment for exploring various healthcare
policies, payment methodologies, etc., and 2) the basis for clinical artificial
intelligence - an AI that can think like a doctor. This approach combines
Markov decision processes and dynamic decision networks to learn from clinical
data and develop complex plans via simulation of alternative sequential
decision paths while capturing the sometimes conflicting, sometimes synergistic
interactions of various components in the healthcare system. It can operate in
partially observable environments (in the case of missing observations or data)
by maintaining belief states about patient health status and functions as an
online agent that plans and re-plans. This framework was evaluated using real
patient data from an electronic health record. Such an AI framework easily
outperforms the current treatment-as-usual (TAU) case-rate/fee-for-service
models of healthcare (Cost per Unit Change: $189 vs. $497) while obtaining a
30-35% increase in patient outcomes. Tweaking certain model parameters further
enhances this advantage, obtaining roughly 50% more improvement for roughly
half the costs. Given careful design and problem formulation, an AI simulation
framework can approximate optimal decisions even in complex and uncertain
environments. Future work is described that outlines potential lines of
research and integration of machine learning algorithms for personalized
medicine.

negative:
This paper presents the first deep reinforcement learning (DRL) framework to
estimate the optimal Dynamic Treatment Regimes from observational medical data.
This framework is more flexible and adaptive for high dimensional action and
state spaces than existing reinforcement learning methods to model real-life
complexity in heterogeneous disease progression and treatment choices, with the
goal of providing doctor and patients the data-driven personalized decision
recommendations. The proposed DRL framework comprises (i) a supervised learning
step to predict the most possible expert actions, and (ii) a deep reinforcement
learning step to estimate the long-term value function of Dynamic Treatment
Regimes. Both steps depend on deep neural networks.
  As a key motivational example, we have implemented the proposed framework on
a data set from the Center for International Bone Marrow Transplant Research
(CIBMTR) registry database, focusing on the sequence of prevention and
treatments for acute and chronic graft versus host disease after
transplantation. In the experimental results, we have demonstrated promising
accuracy in predicting human experts' decisions, as well as the high expected
reward function in the DRL-based dynamic treatment regimes.

negative:
Reinforcement Learning (RL) has recently been applied to sequential
estimation and prediction problems identifying and developing hypothetical
treatment strategies for septic patients, with a particular focus on offline
learning with observational data. In practice, successful RL relies on
informative latent states derived from sequential observations to develop
optimal treatment strategies. To date, how best to construct such states in a
healthcare setting is an open question. In this paper, we perform an empirical
study of several information encoding architectures using data from septic
patients in the MIMIC-III dataset to form representations of a patient state.
We evaluate the impact of representation dimension, correlations with
established acuity scores, and the treatment policies derived from them. We
find that sequentially formed state representations facilitate effective policy
learning in batch settings, validating a more thoughtful approach to
representation learning that remains faithful to the sequential and partial
nature of healthcare data.

negative:
Personalized predictive medicine necessitates the modeling of patient illness
and care processes, which inherently have long-term temporal dependencies.
Healthcare observations, recorded in electronic medical records, are episodic
and irregular in time. We introduce DeepCare, an end-to-end deep dynamic neural
network that reads medical records, stores previous illness history, infers
current illness states and predicts future medical outcomes. At the data level,
DeepCare represents care episodes as vectors in space, models patient health
state trajectories through explicit memory of historical records. Built on Long
Short-Term Memory (LSTM), DeepCare introduces time parameterizations to handle
irregular timed events by moderating the forgetting and consolidation of memory
cells. DeepCare also incorporates medical interventions that change the course
of illness and shape future medical risk. Moving up to the health state level,
historical and present health states are then aggregated through multiscale
temporal pooling, before passing through a neural network that estimates future
outcomes. We demonstrate the efficacy of DeepCare for disease progression
modeling, intervention recommendation, and future risk prediction. On two
important cohorts with heavy social and economic burden -- diabetes and mental
health -- the results show improved modeling and risk prediction accuracy.

negative:
As a subfield of machine learning, reinforcement learning (RL) aims at
empowering one's capabilities in behavioural decision making by using
interaction experience with the world and an evaluative feedback. Unlike
traditional supervised learning methods that usually rely on one-shot,
exhaustive and supervised reward signals, RL tackles with sequential decision
making problems with sampled, evaluative and delayed feedback simultaneously.
Such distinctive features make RL technique a suitable candidate for developing
powerful solutions in a variety of healthcare domains, where diagnosing
decisions or treatment regimes are usually characterized by a prolonged and
sequential procedure. This survey discusses the broad applications of RL
techniques in healthcare domains, in order to provide the research community
with systematic understanding of theoretical foundations, enabling methods and
techniques, existing challenges, and new insights of this emerging paradigm. By
first briefly examining theoretical foundations and key techniques in RL
research from efficient and representational directions, we then provide an
overview of RL applications in healthcare domains ranging from dynamic
treatment regimes in chronic diseases and critical care, automated medical
diagnosis from both unstructured and structured clinical data, as well as many
other control or scheduling domains that have infiltrated many aspects of a
healthcare system. Finally, we summarize the challenges and open issues in
current research, and point out some potential solutions and directions for
future research.

negative:
Management of chronic diseases such as heart failure, diabetes, and chronic
obstructive pulmonary disease (COPD) is a major problem in health care. A
standard approach that the medical community has devised to manage widely
prevalent chronic diseases such as chronic heart failure (CHF) is to have a
committee of experts develop guidelines that all physicians should follow.
These guidelines typically consist of a series of complex rules that make
recommendations based on a patient's information. Due to their complexity,
often the guidelines are either ignored or not complied with at all, which can
result in poor medical practices. It is not even clear whether it is humanly
possible to follow these guidelines due to their length and complexity. In the
case of CHF management, the guidelines run nearly 80 pages. In this paper we
describe a physician-advisory system for CHF management that codes the entire
set of clinical practice guidelines for CHF using answer set programming. Our
approach is based on developing reasoning templates (that we call knowledge
patterns) and using these patterns to systemically code the clinical guidelines
for CHF as ASP rules. Use of the knowledge patterns greatly facilitates the
development of our system. Given a patient's medical information, our system
generates a recommendation for treatment just as a human physician would, using
the guidelines. Our system will work even in the presence of incomplete
information. Our work makes two contributions: (i) it shows that highly complex
guidelines can be successfully coded as ASP rules, and (ii) it develops a
series of knowledge patterns that facilitate the coding of knowledge expressed
in a natural language and that can be used for other application domains. This
paper is under consideration for acceptance in TPLP.

negative:
Objective: Electronic medical records (EMRs) contain an amount of medical
knowledge which can be used for clinical decision support (CDS). Our objective
is a general system that can extract and represent these knowledge contained in
EMRs to support three CDS tasks: test recommendation, initial diagnosis, and
treatment plan recommendation, with the given condition of one patient.
Methods: We extracted four kinds of medical entities from records and
constructed an EMR-based medical knowledge network (EMKN), in which nodes are
entities and edges reflect their co-occurrence in a single record. Three
bipartite subgraphs (bi-graphs) were extracted from the EMKN to support each
task. One part of the bi-graph was the given condition (e.g., symptoms), and
the other was the condition to be inferred (e.g., diseases). Each bi-graph was
regarded as a Markov random field to support the inference. Three lazy energy
functions and one parameter-based energy function were proposed, as well as two
knowledge representation learning-based energy functions, which can provide a
distributed representation of medical entities. Three measures were utilized
for performance evaluation. Results: On the initial diagnosis task, 80.11% of
the test records identified at least one correct disease from top 10
candidates. Test and treatment recommendation results were 87.88% and 92.55%,
respectively. These results altogether indicate that the proposed system
outperformed the baseline methods. The distributed representation of medical
entities does reflect similarity relationships in regards to knowledge level.
Conclusion: Combining EMKN and MRF is an effective approach for general medical
knowledge representation and inference. Different tasks, however, require
designing their energy functions individually.

negative:
There is a clear need to involve patients in medical decisions. However,
cognitive psychological research has highlighted the cognitive limitations of
humans with respect to 1. Probabilistic assessment of the patient state and of
potential outcomes of various decisions, 2. Elicitation of the patient utility
function, and 3. Integration of the probabilistic knowledge and of patient
preferences to determine the optimal strategy. Therefore, without adequate
computational support, current shared decision models have severe ethical
deficiencies. An informed consent model unfairly transfers the responsibility
to a patient who does not have the necessary knowledge, nor the integration
capability. A paternalistic model endows with exaggerated power a physician who
might not be aware of the patient preferences, is prone to multiple cognitive
biases, and whose computational integration capability is bounded. Recent
progress in Artificial Intelligence suggests adding a third agent: a computer,
in all deliberative medical decisions: Non emergency medical decisions in which
more than one alternative exists, the patient preferences can be elicited, the
therapeutic alternatives might be influenced by these preferences, medical
knowledge exists regarding the likelihood of the decision outcomes, and there
is sufficient decision time. Ethical physicians should exploit computational
decision support technologies, neither making the decisions solely on their
own, nor shirking their duty and shifting the responsibility to patients in the
name of informed consent. The resulting three way (patient, care provider,
computer) human machine model that we suggest emphasizes the patient
preferences, the physician knowledge, and the computational integration of both
aspects, does not diminish the physician role, but rather brings out the best
in human and machine.

negative:
In this document, we explore in more detail our published work (Komorowski,
Celi, Badawi, Gordon, & Faisal, 2018) for the benefit of the AI in Healthcare
research community. In the above paper, we developed the AI Clinician system,
which demonstrated how reinforcement learning could be used to make useful
recommendations towards optimal treatment decisions from intensive care data.
Since publication a number of authors have reviewed our work (e.g. Abbasi,
2018; Bos, Azoulay, & Martin-Loeches, 2019; Saria, 2018). Given the difference
of our framework to previous work, the fact that we are bridging two very
different academic communities (intensive care and machine learning) and that
our work has impact on a number of other areas with more traditional
computer-based approaches (biosignal processing and control, biomedical
engineering), we are providing here additional details on our recent
publication.

negative:
Sepsis is a leading cause of mortality in intensive care units and costs
hospitals billions annually. Treating a septic patient is highly challenging,
because individual patients respond very differently to medical interventions
and there is no universally agreed-upon treatment for sepsis. In this work, we
propose an approach to deduce treatment policies for septic patients by using
continuous state-space models and deep reinforcement learning. Our model learns
clinically interpretable treatment policies, similar in important aspects to
the treatment policies of physicians. The learned policies could be used to aid
intensive care clinicians in medical decision making and improve the likelihood
of patient survival.

negative:
Sepsis is a leading cause of mortality in intensive care units (ICUs) and
costs hospitals billions annually. Treating a septic patient is highly
challenging, because individual patients respond very differently to medical
interventions and there is no universally agreed-upon treatment for sepsis.
Understanding more about a patient's physiological state at a given time could
hold the key to effective treatment policies. In this work, we propose a new
approach to deduce optimal treatment policies for septic patients by using
continuous state-space models and deep reinforcement learning. Learning
treatment policies over continuous spaces is important, because we retain more
of the patient's physiological information. Our model is able to learn
clinically interpretable treatment policies, similar in important aspects to
the treatment policies of physicians. Evaluating our algorithm on past ICU
patient data, we find that our model could reduce patient mortality in the
hospital by up to 3.6% over observed clinical policies, from a baseline
mortality of 13.7%. The learned treatment policies could be used to aid
intensive care clinicians in medical decision making and improve the likelihood
of patient survival.

negative:
Finding an effective medical treatment often requires a search by trial and
error. Making this search more efficient by minimizing the number of
unnecessary trials could lower both costs and patient suffering. We formalize
this problem as learning a policy for finding a near-optimal treatment in a
minimum number of trials using a causal inference framework. We give a
model-based dynamic programming algorithm which learns from observational data
while being robust to unmeasured confounding. To reduce time complexity, we
suggest a greedy algorithm which bounds the near-optimality constraint. The
methods are evaluated on synthetic and real-world healthcare data and compared
to model-free reinforcement learning. We find that our methods compare
favorably to the model-free baseline while offering a more transparent
trade-off between search time and treatment efficacy.

negative:
Decision makers, such as doctors and judges, make crucial decisions such as
recommending treatments to patients, and granting bails to defendants on a
daily basis. Such decisions typically involve weighting the potential benefits
of taking an action against the costs involved. In this work, we aim to
automate this task of learning \emph{cost-effective, interpretable and
actionable treatment regimes}. We formulate this as a problem of learning a
decision list -- a sequence of if-then-else rules -- which maps characteristics
of subjects (eg., diagnostic test results of patients) to treatments. We
propose a novel objective to construct a decision list which maximizes outcomes
for the population, and minimizes overall costs. We model the problem of
learning such a list as a Markov Decision Process (MDP) and employ a variant of
the Upper Confidence Bound for Trees (UCT) strategy which leverages customized
checks for pruning the search space effectively. Experimental results on real
world observational data capturing judicial bail decisions and treatment
recommendations for asthma patients demonstrate the effectiveness of our
approach.

negative:
Clinical decision support tools rooted in machine learning and optimization
can provide significant value to healthcare providers, including through better
management of intensive care units. In particular, it is important that the
patient discharge task addresses the nuanced trade-off between decreasing a
patient's length of stay (and associated hospitalization costs) and the risk of
readmission or even death following the discharge decision. This work
introduces an end-to-end general framework for capturing this trade-off to
recommend optimal discharge timing decisions given a patient's electronic
health records. A data-driven approach is used to derive a parsimonious,
discrete state space representation that captures a patient's physiological
condition. Based on this model and a given cost function, an infinite-horizon
discounted Markov decision process is formulated and solved numerically to
compute an optimal discharge policy, whose value is assessed using off-policy
evaluation strategies. Extensive numerical experiments are performed to
validate the proposed framework using real-life intensive care unit patient
data.

negative:
In mobile health (mHealth) smart devices deliver behavioral treatments
repeatedly over time to a user with the goal of helping the user adopt and
maintain healthy behaviors. Reinforcement learning appears ideal for learning
how to optimally make these sequential treatment decisions. However,
significant challenges must be overcome before reinforcement learning can be
effectively deployed in a mobile healthcare setting. In this work we are
concerned with the following challenges: 1) individuals who are in the same
context can exhibit differential response to treatments 2) only a limited
amount of data is available for learning on any one individual, and 3)
non-stationary responses to treatment. To address these challenges we
generalize Thompson-Sampling bandit algorithms to develop IntelligentPooling.
IntelligentPooling learns personalized treatment policies thus addressing
challenge one. To address the second challenge, IntelligentPooling updates each
user's degree of personalization while making use of available data on other
users to speed up learning. Lastly, IntelligentPooling allows responsivity to
vary as a function of a user's time since beginning treatment, thus addressing
challenge three. We show that IntelligentPooling achieves an average of 26%
lower regret than state-of-the-art. We demonstrate the promise of this approach
and its ability to learn from even a small group of users in a live clinical
trial.

negative:
In both the fields of computer science and medicine there is very strong
interest in developing personalized treatment policies for patients who have
variable responses to treatments. In particular, I aim to find an optimal
personalized treatment policy which is a non-deterministic function of the
patient specific covariate data that maximizes the expected survival time or
clinical outcome. I developed an algorithmic framework to solve multistage
decision problem with a varying number of stages that are subject to censoring
in which the "rewards" are expected survival times. In specific, I developed a
novel Q-learning algorithm that dynamically adjusts for these parameters.
Furthermore, I found finite upper bounds on the generalized error of the
treatment paths constructed by this algorithm. I have also shown that when the
optimal Q-function is an element of the approximation space, the anticipated
survival times for the treatment regime constructed by the algorithm will
converge to the optimal treatment path. I demonstrated the performance of the
proposed algorithmic framework via simulation studies and through the analysis
of chronic depression data and a hypothetical clinical trial. The censored
Q-learning algorithm I developed is more effective than the state of the art
clinical decision support systems and is able to operate in environments when
many covariate parameters may be unobtainable or censored.

negative:
Automated medical prognosis has gained interest as artificial intelligence
evolves and the potential for computer-aided medicine becomes evident.
Nevertheless, it is challenging to design an effective system that, given a
patient's medical history, is able to predict probable future conditions.
Previous works, mostly carried out over private datasets, have tackled the
problem by using artificial neural network architectures that cannot deal with
low-cardinality datasets, or by means of non-generalizable inference
approaches. We introduce a Deep Learning architecture whose design results from
an intensive experimental process. The final architecture is based on two
parallel Minimal Gated Recurrent Unit networks working in bi-directional
manner, which was extensively tested with the open-access Mimic-III dataset.
Our results demonstrate significant improvements in automated medical
prognosis, as measured with Recall@k. We summarize our experience as a set of
relevant insights for the design of Deep Learning architectures. Our work
improves the performance of computer-aided medicine and can serve as a guide in
designing artificial neural networks used in prediction tasks.

negative:
Our aim is to establish a framework where reinforcement learning (RL) of
optimizing interventions retrospectively allows us a regulatory compliant
pathway to prospective clinical testing of the learned policies in a clinical
deployment. We focus on infections in intensive care units which are one of the
major causes of death and difficult to treat because of the complex and opaque
patient dynamics, and the clinically debated, highly-divergent set of
intervention policies required by each individual patient, yet intensive care
units are naturally data rich. In our work, we build on RL approaches in
healthcare ("AI Clinicians"), and learn off-policy continuous dosing policy of
pharmaceuticals for sepsis treatment using historical intensive care data under
partially observable MDPs (POMDPs). POMPDs capture uncertainty in patient state
better by taking in all historical information, yielding an efficient
representation, which we investigate through ablations. We compensate for the
lack of exploration in our retrospective data by evaluating each encountered
state with a best-first tree search. We mitigate state distributional shift by
optimizing our policy in the vicinity of the clinicians' compound policy.
Crucially, we evaluate our model recommendations using not only conventional
policy evaluations but a novel framework that incorporates human experts: a
model-agnostic pre-clinical evaluation method to estimate the accuracy and
uncertainty of clinician's decisions versus our system recommendations when
confronted with the same individual patient history ("shadow mode").

negative:
Modern healthcare is ripe for disruption by AI. A game changer would be
automatic understanding the latent processes from electronic medical records,
which are being collected for billions of people worldwide. However, these
healthcare processes are complicated by the interaction between at least three
dynamic components: the illness which involves multiple diseases, the care
which involves multiple treatments, and the recording practice which is biased
and erroneous. Existing methods are inadequate in capturing the dynamic
structure of care. We propose Resset, an end-to-end recurrent model that reads
medical record and predicts future risk. The model adopts the algebraic view in
that discrete medical objects are embedded into continuous vectors lying in the
same space. We formulate the problem as modeling sequences of sets, a novel
setting that have rarely, if not, been addressed. Within Resset, the bag of
diseases recorded at each clinic visit is modeled as function of sets. The same
hold for the bag of treatments. The interaction between the disease bag and the
treatment bag at a visit is modeled in several, one of which as residual of
diseases minus the treatments. Finally, the health trajectory, which is a
sequence of visits, is modeled using a recurrent neural network. We report
results on over a hundred thousand hospital visits by patients suffered from
two costly chronic diseases -- diabetes and mental health. Resset shows
promises in multiple predictive tasks such as readmission prediction,
treatments recommendation and diseases progression.

negative:
While reinforcement learning (RL) has proven to be the approach of choice for
tackling many complex problems, it remains challenging to develop and deploy RL
agents in real-life scenarios successfully. This paper presents pH-RL
(personalization in e-Health with RL) a general RL architecture for
personalization to bring RL to health practice. pH-RL allows for various levels
of personalization in health applications and allows for online and batch
learning. Furthermore, we provide a general-purpose implementation framework
that can be integrated with various healthcare applications. We describe a
step-by-step guideline for the successful deployment of RL policies in a mobile
application. We implemented our open-source RL architecture and integrated it
with the MoodBuster mobile application for mental health to provide messages to
increase daily adherence to the online therapeutic modules. We then performed a
comprehensive study with human participants over a sustained period. Our
experimental results show that the developed policies learn to select
appropriate actions consistently using only a few days' worth of data.
Furthermore, we empirically demonstrate the stability of the learned policies
during the study.

negative:
Choosing the best treatment-plan for each individual patient requires
accurate forecasts of their outcome trajectories as a function of the
treatment, over time. While large observational data sets constitute rich
sources of information to learn from, they also contain biases as treatments
are rarely assigned randomly in practice. To provide accurate and unbiased
forecasts, we introduce the Disentangled Counterfactual Recurrent Network
(DCRN), a novel sequence-to-sequence architecture that estimates treatment
outcomes over time by learning representations of patient histories that are
disentangled into three separate latent factors: a treatment factor,
influencing only treatment selection; an outcome factor, influencing only the
outcome; and a confounding factor, influencing both. With an architecture that
is completely inspired by the causal structure of treatment influence over
time, we advance forecast accuracy and disease understanding, as our
architecture allows for practitioners to infer which patient features influence
which part in a patient's trajectory, contrasting other approaches in this
domain. We demonstrate that DCRN outperforms current state-of-the-art methods
in forecasting treatment responses, on both real and simulated data.

negative:
Objectives: Electronic health records (EHRs) are only a first step in
capturing and utilizing health-related data - the challenge is turning that
data into useful information. Furthermore, EHRs are increasingly likely to
include data relating to patient outcomes, functionality such as clinical
decision support, and genetic information as well, and, as such, can be seen as
repositories of increasingly valuable information about patients' health
conditions and responses to treatment over time. Methods: We describe a case
study of 423 patients treated by Centerstone within Tennessee and Indiana in
which we utilized electronic health record data to generate predictive
algorithms of individual patient treatment response. Multiple models were
constructed using predictor variables derived from clinical, financial and
geographic data. Results: For the 423 patients, 101 deteriorated, 223 improved
and in 99 there was no change in clinical condition. Based on modeling of
various clinical indicators at baseline, the highest accuracy in predicting
individual patient response ranged from 70-72% within the models tested. In
terms of individual predictors, the Centerstone Assessment of Recovery Level -
Adult (CARLA) baseline score was most significant in predicting outcome over
time (odds ratio 4.1 + 2.27). Other variables with consistently significant
impact on outcome included payer, diagnostic category, location and provision
of case management services. Conclusions: This approach represents a promising
avenue toward reducing the current gap between research and practice across
healthcare, developing data-driven clinical decision support based on
real-world populations, and serving as a component of embedded clinical
artificial intelligences that "learn" over time.

negative:
Many healthcare decisions involve navigating through a multitude of treatment
options in a sequential and iterative manner to find an optimal treatment
pathway with the goal of an optimal patient outcome. Such optimization problems
may be amenable to reinforcement learning. A reinforcement learning agent could
be trained to provide treatment recommendations for physicians, acting as a
decision support tool. However, a number of difficulties arise when using RL
beyond benchmark environments, such as specifying the reward function, choosing
an appropriate state representation and evaluating the learned policy.

negative:
Automatically reasoning with conflicting generic clinical guidelines is a
burning issue in patient-centric medical reasoning where patient-specific
conditions and goals need to be taken into account. It is even more challenging
in the presence of preferences such as patient's wishes and clinician's
priorities over goals. We advance a structured argumentation formalism for
reasoning with conflicting clinical guidelines, patient-specific information
and preferences. Our formalism integrates assumption-based reasoning and
goal-driven selection among reasoning outcomes. Specifically, we assume
applicability of guideline recommendations concerning the generic goal of
patient well-being, resolve conflicts among recommendations using patient's
conditions and preferences, and then consider prioritised patient-centered
goals to yield non-conflicting, goal-maximising and preference-respecting
recommendations. We rely on the state-of-the-art Transition-based Medical
Recommendation model for representing guideline recommendations and augment it
with context given by the patient's conditions, goals, as well as preferences
over recommendations and goals. We establish desirable properties of our
approach in terms of sensitivity to recommendation conflicts and patient
context.

negative:
Much attention has been devoted recently to the development of machine
learning algorithms with the goal of improving treatment policies in
healthcare. Reinforcement learning (RL) is a sub-field within machine learning
that is concerned with learning how to make sequences of decisions so as to
optimize long-term effects. Already, RL algorithms have been proposed to
identify decision-making strategies for mechanical ventilation, sepsis
management and treatment of schizophrenia. However, before implementing
treatment policies learned by black-box algorithms in high-stakes clinical
decision problems, special care must be taken in the evaluation of these
policies.
  In this document, our goal is to expose some of the subtleties associated
with evaluating RL algorithms in healthcare. We aim to provide a conceptual
starting point for clinical and computational researchers to ask the right
questions when designing and evaluating algorithms for new ways of treating
patients. In the following, we describe how choices about how to summarize a
history, variance of statistical estimators, and confounders in more ad-hoc
measures can result in unreliable, even misleading estimates of the quality of
a treatment policy. We also provide suggestions for mitigating these
effects---for while there is much promise for mining observational health data
to uncover better treatment policies, evaluation must be performed
thoughtfully.

negative:
We consider a reinforcement learning (RL) setting in which the agent
interacts with a sequence of episodic MDPs. At the start of each episode the
agent has access to some side-information or context that determines the
dynamics of the MDP for that episode. Our setting is motivated by applications
in healthcare where baseline measurements of a patient at the start of a
treatment episode form the context that may provide information about how the
patient might respond to treatment decisions. We propose algorithms for
learning in such Contextual Markov Decision Processes (CMDPs) under an
assumption that the unobserved MDP parameters vary smoothly with the observed
context. We also give lower and upper PAC bounds under the smoothness
assumption. Because our lower bound has an exponential dependence on the
dimension, we consider a tractable linear setting where the context is used to
create linear combinations of a finite set of MDPs. For the linear setting, we
give a PAC learning algorithm based on KWIK learning techniques.

negative:
Document classification is a challenging task with important applications.
The deep learning approaches to the problem have gained much attention
recently. Despite the progress, the proposed models do not incorporate the
knowledge of the document structure in the architecture efficiently and not
take into account the contexting importance of words and sentences. In this
paper, we propose a new approach based on a combination of convolutional neural
networks, gated recurrent units, and attention mechanisms for document
classification tasks. The main contribution of this work is the use of
convolution layers to extract more meaningful, generalizable and abstract
features by the hierarchical representation. The proposed method in this paper
improves the results of the current attention-based approaches for document
classification.

negative:
Identifying when to give treatments to patients and how to select among
multiple treatments over time are important medical problems with a few
existing solutions. In this paper, we introduce the Counterfactual Recurrent
Network (CRN), a novel sequence-to-sequence model that leverages the
increasingly available patient observational data to estimate treatment effects
over time and answer such medical questions. To handle the bias from
time-varying confounders, covariates affecting the treatment assignment policy
in the observational data, CRN uses domain adversarial training to build
balancing representations of the patient history. At each timestep, CRN
constructs a treatment invariant representation which removes the association
between patient history and treatment assignments and thus can be reliably used
for making counterfactual predictions. On a simulated model of tumour growth,
with varying degree of time-dependent confounding, we show how our model
achieves lower error in estimating counterfactuals and in choosing the correct
treatment and timing of treatment than current state-of-the-art methods.

negative:
Medical head CT-scan imaging has been successfully combined with deep
learning for medical diagnostics of head diseases and lesions[1]. State of the
art classification models and algorithms for this task usually are based on 3d
convolution layers for volumetric data on a supervised learning setting (1
input volume, 1 prediction per patient) or 2d convolution layers in a
supervised setting (1 input image, 1 prediction per image). However a very
common scenario in developing countries is to have the volume metadata lost due
multiple reasons for example formatting conversion in images (for example
.dicom to jpg), in this scenario the doctor analyses the collection of images
and then emits a single diagnostic for the patient (with possibly an unfixed
and variable number of images per patient) , this prevents it from being
possible to use state of the art 3d models, but also is not possible to convert
it to a supervised problem in a (1 image,1 diagnostic) setting because
different angles or positions of the images for a single patient may not
contain the disease or lesion. In this study we propose a solution for this
scenario by combining 2d convolutional[2] models with sequence models which
generate a prediction only after all images have been processed by the model
for a given patient \(i\), this creates a multi-image to single-diagnostic
setting \(y^i=f(x_1,x_2,..,x_n)\) where \(n\) may be different between
patients. The experimental results demonstrate that it is possible to get a
multi-image to single diagnostic model which mimics human doctor diagnostic
process: evaluate the collection of patient images and then use important
information in memory to decide a single diagnostic for the patient.

negative:
Healthcare data suffers from both noise and lack of ground truth. The cost of
data increases as it is cleaned and annotated in healthcare. Unlike other data
sets, medical data annotation, which is critical to accurate ground truth,
requires medical domain expertise for a better patient outcome. In this work,
we report on the use of reinforcement learning to mimic the decision making
process of annotators for medical events, to automate annotation and labelling.
The reinforcement agent learns to annotate alarm data based on annotations done
by an expert. Our method shows promising results on medical alarm data sets. We
trained DQN and A2C agents using the data from monitoring devices annotated by
an expert. Initial results from these RL agents learning the expert annotation
behavior are promising. The A2C agent performs better in terms of learning the
sparse events in a given state, thereby choosing more right actions compared to
DQN agent. To the best of our knowledge, this is the first reinforcement
learning application for the automation of medical events annotation, which has
far-reaching practical use.

negative:
Recent advances in personalized recommendation have sparked great interest in
the exploitation of rich structured information provided by knowledge graphs.
Unlike most existing approaches that only focus on leveraging knowledge graphs
for more accurate recommendation, we perform explicit reasoning with knowledge
for decision making so that the recommendations are generated and supported by
an interpretable causal inference procedure. To this end, we propose a method
called Policy-Guided Path Reasoning (PGPR), which couples recommendation and
interpretability by providing actual paths in a knowledge graph. Our
contributions include four aspects. We first highlight the significance of
incorporating knowledge graphs into recommendation to formally define and
interpret the reasoning process. Second, we propose a reinforcement learning
(RL) approach featuring an innovative soft reward strategy, user-conditional
action pruning and a multi-hop scoring function. Third, we design a
policy-guided graph search algorithm to efficiently and effectively sample
reasoning paths for recommendation. Finally, we extensively evaluate our method
on several large-scale real-world benchmark datasets, obtaining favorable
results compared with state-of-the-art methods.

negative:
Randomized controlled trials typically analyze the effectiveness of
treatments with the goal of making treatment recommendations for patient
subgroups. With the advance of electronic health records, a great variety of
data has been collected in clinical practice, enabling the evaluation of
treatments and treatment policies based on observational data. In this paper,
we focus on learning individualized treatment rules (ITRs) to derive a
treatment policy that is expected to generate a better outcome for an
individual patient. In our framework, we cast ITRs learning as a contextual
bandit problem and minimize the expected risk of the treatment policy. We
conduct experiments with the proposed framework both in a simulation study and
based on a real-world dataset. In the latter case, we apply our proposed method
to learn the optimal ITRs for the administration of intravenous (IV) fluids and
vasopressors (VP). Based on various offline evaluation methods, we could show
that the policy derived in our framework demonstrates better performance
compared to both the physicians and other baselines, including a simple
treatment prediction approach. As a long-term goal, our derived policy might
eventually lead to better clinical guidelines for the administration of IV and
VP.

negative:
Evaluating the clinical similarities between pairwise patients is a
fundamental problem in healthcare informatics. A proper patient similarity
measure enables various downstream applications, such as cohort study and
treatment comparative effectiveness research. One major carrier for conducting
patient similarity research is Electronic Health Records(EHRs), which are
usually heterogeneous, longitudinal, and sparse. Though existing studies on
learning patient similarity from EHRs have shown being useful in solving real
clinical problems, their applicability is limited due to the lack of medical
interpretations. Moreover, most previous methods assume a vector-based
representation for patients, which typically requires aggregation of medical
events over a certain time period. As a consequence, temporal information will
be lost. In this paper, we propose a patient similarity evaluation framework
based on the temporal matching of longitudinal patient EHRs. Two efficient
methods are presented, unsupervised and supervised, both of which preserve the
temporal properties in EHRs. The supervised scheme takes a convolutional neural
network architecture and learns an optimal representation of patient clinical
records with medical concept embedding. The empirical results on real-world
clinical data demonstrate substantial improvement over the baselines. We make
our code and sample data available for further study.

negative:
Low attendance levels in medical appointments have been associated with poor
health outcomes and efficiency problems for service providers. To address this
problem, healthcare managers could aim at improving attendance levels or
minimizing the operational impact of no-shows by adapting resource allocation
policies. However, given the uncertainty of patient behaviour, generating
relevant information regarding no-show probabilities could support the
decision-making process for both approaches. In this context many researchers
have used multiple regression models to identify patient and appointment
characteristics than can be used as good predictors for no-show probabilities.
This work develops a Decision Support System (DSS) to support the
implementation of strategies to encourage attendance, for a preventive care
program targeted at underserved communities in Bogot\'a, Colombia. Our
contribution to literature is threefold. Firstly, we assess the effectiveness
of different machine learning approaches to improve the accuracy of regression
models. In particular, Random Forest and Neural Networks are used to model the
problem accounting for non-linearity and variable interactions. Secondly, we
propose a novel use of Layer-wise Relevance Propagation in order to improve the
explainability of neural network predictions and obtain insights from the
modelling step. Thirdly, we identify variables explaining no-show probabilities
in a developing context and study its policy implications and potential for
improving healthcare access. In addition to quantifying relationships reported
in previous studies, we find that income and neighbourhood crime statistics
affect no-show probabilities. Our results will support patient prioritization
in a pilot behavioural intervention and will inform appointment planning
decisions.

negative:
Machine learning has successfully framed many sequential decision making
problems as either supervised prediction, or optimal decision-making policy
identification via reinforcement learning. In data-constrained offline
settings, both approaches may fail as they assume fully optimal behavior or
rely on exploring alternatives that may not exist. We introduce an inherently
different approach that identifies possible "dead-ends" of a state space. We
focus on the condition of patients in the intensive care unit, where a "medical
dead-end" indicates that a patient will expire, regardless of all potential
future treatment sequences. We postulate "treatment security" as avoiding
treatments with probability proportional to their chance of leading to
dead-ends, present a formal proof, and frame discovery as an RL problem. We
then train three independent deep neural models for automated state
construction, dead-end discovery and confirmation. Our empirical results
discover that dead-ends exist in real clinical data among septic patients, and
further reveal gaps between secure treatments and those that were administered.

negative:
Objective: To combine medical knowledge and medical data to interpretably
predict the risk of disease. Methods: We formulated the disease prediction task
as a random walk along a knowledge graph (KG). Specifically, we build a KG to
record relationships between diseases and risk factors according to validated
medical knowledge. Then, a mathematical object walks along the KG. It starts
walking at a patient entity, which connects the KG based on the patient current
diseases or risk factors and stops at a disease entity, which represents the
predicted disease. The trajectory generated by the object represents an
interpretable disease progression path of the given patient. The dynamics of
the object are controlled by a policy-based reinforcement learning (RL) module,
which is trained by electronic health records (EHRs). Experiments: We utilized
two real-world EHR datasets to evaluate the performance of our model. In the
disease prediction task, our model achieves 0.743 and 0.639 in terms of macro
area under the curve (AUC) in predicting 53 circulation system diseases in the
two datasets, respectively. This performance is comparable to the commonly used
machine learning (ML) models in medical research. In qualitative analysis, our
clinical collaborator reviewed the disease progression paths generated by our
model and advocated their interpretability and reliability. Conclusion:
Experimental results validate the proposed model in interpretably evaluating
and optimizing disease prediction. Significance: Our work contributes to
leveraging the potential of medical knowledge and medical data jointly for
interpretable prediction tasks.

negative:
Opioids are the preferred medications for the treatment of pain in the
intensive care unit. While undertreatment leads to unrelieved pain and poor
clinical outcomes, excessive use of opioids puts patients at risk of
experiencing multiple adverse effects. In this work, we present a sequential
decision making framework for opioid dosing based on deep reinforcement
learning. It provides real-time clinically interpretable dosing
recommendations, personalized according to each patient's evolving pain and
physiological condition. We focus on morphine, one of the most commonly
prescribed opioids. To train and evaluate the model, we used retrospective data
from the publicly available MIMIC-3 database. Our results demonstrate that
reinforcement learning may be used to aid decision making in the intensive care
setting by providing personalized pain management interventions.

negative:
A treatment regime is a function that maps individual patient information to
a recommended treatment, hence explicitly incorporating the heterogeneity in
need for treatment across individuals. Patient responses are dichotomous and
can be predicted through an unknown relationship that depends on the patient
information and the selected treatment. The goal is to find the treatments that
lead to the best patient responses on average. Each experiment is expensive,
forcing us to learn the most from each experiment. We adopt a Bayesian approach
both to incorporate possible prior information and to update our treatment
regime continuously as information accrues, with the potential to allow smaller
yet more informative trials and for patients to receive better treatment. By
formulating the problem as contextual bandits, we introduce a knowledge
gradient policy to guide the treatment assignment by maximizing the expected
value of information, for which an approximation method is used to overcome
computational challenges. We provide a detailed study on how to make sequential
medical decisions under uncertainty to reduce health care costs on a real world
knee replacement dataset. We use clustering and LASSO to deal with the
intrinsic sparsity in health datasets. We show experimentally that even though
the problem is sparse, through careful selection of physicians (versus picking
them at random), we can significantly improve the success rates.

negative:
We propose to use boosted regression trees as a way to compute
human-interpretable solutions to reinforcement learning problems. Boosting
combines several regression trees to improve their accuracy without
significantly reducing their inherent interpretability. Prior work has focused
independently on reinforcement learning and on interpretable machine learning,
but there has been little progress in interpretable reinforcement learning. Our
experimental results show that boosted regression trees compute solutions that
are both interpretable and match the quality of leading reinforcement learning
methods.

negative:
Doctors often rely on their past experience in order to diagnose patients.
For a doctor with enough experience, almost every patient would have
similarities to key cases seen in the past, and each new patient could be
viewed as a mixture of these key past cases. Because doctors often tend to
reason this way, an efficient computationally aided diagnostic tool that thinks
in the same way might be helpful in locating key past cases of interest that
could assist with diagnosis. This article develops a novel mathematical model
to mimic the type of logical thinking that physicians use when considering past
cases. The proposed model can also provide physicians with explanations that
would be similar to the way they would naturally reason about cases. The
proposed method is designed to yield predictive accuracy, computational
efficiency, and insight into medical data; the key element is the insight into
medical data, in some sense we are automating a complicated process that
physicians might perform manually. We finally implemented the result of this
work on two publicly available healthcare datasets, for heart disease
prediction and breast cancer prediction.

negative:
Cardiovascular diseases and heart failures in particular are the main cause
of non-communicable disease mortality in the world. Constant patient monitoring
enables better medical treatment as it allows practitioners to react on time
and provide the appropriate treatment. Telemedicine can provide constant remote
monitoring so patients can stay in their homes, only requiring medical sensing
equipment and network connections. A limiting factor for telemedical centers is
the amount of patients that can be monitored simultaneously. We aim to increase
this amount by implementing a decision support system. This paper investigates
a machine learning model to estimate a risk score based on patient vital
parameters that allows sorting all cases every day to help practitioners focus
their limited capacities on the most severe cases. The model we propose reaches
an AUCROC of 0.84, whereas the baseline rule-based model reaches an AUCROC of
0.73. Our results indicate that the usage of deep learning to improve the
efficiency of telemedical centers is feasible. This way more patients could
benefit from better health-care through remote monitoring.

negative:
During the first wave of COVID-19, hospitals were overwhelmed with the high
number of admitted patients. An accurate prediction of the most likely
individual disease progression can improve the planning of limited resources
and finding the optimal treatment for patients. However, when dealing with a
newly emerging disease such as COVID-19, the impact of patient- and
disease-specific factors (e.g. body weight or known co-morbidities) on the
immediate course of disease is by and large unknown. In the case of COVID-19,
the need for intensive care unit (ICU) admission of pneumonia patients is often
determined only by acute indicators such as vital signs (e.g. breathing rate,
blood oxygen levels), whereas statistical analysis and decision support systems
that integrate all of the available data could enable an earlier prognosis. To
this end, we propose a holistic graph-based approach combining both imaging and
non-imaging information. Specifically, we introduce a multimodal similarity
metric to build a population graph for clustering patients and an image-based
end-to-end Graph Attention Network to process this graph and predict the
COVID-19 patient outcomes: admission to ICU, need for ventilation and
mortality. Additionally, the network segments chest CT images as an auxiliary
task and extracts image features and radiomics for feature fusion with the
available metadata. Results on a dataset collected in Klinikum rechts der Isar
in Munich, Germany show that our approach outperforms single modality and
non-graph baselines. Moreover, our clustering and graph attention allow for
increased understanding of the patient relationships within the population
graph and provide insight into the network's decision-making process.

negative:
For mental disorders, patients' underlying mental states are non-observed
latent constructs which have to be inferred from observed multi-domain
measurements such as diagnostic symptoms and patient functioning scores.
Additionally, substantial heterogeneity in the disease diagnosis between
patients needs to be addressed for optimizing individualized treatment policy
in order to achieve precision medicine. To address these challenges, we propose
an integrated learning framework that can simultaneously learn patients'
underlying mental states and recommend optimal treatments for each individual.
This learning framework is based on the measurement theory in psychiatry for
modeling multiple disease diagnostic measures as arising from the underlying
causes (true mental states). It allows incorporation of the multivariate pre-
and post-treatment outcomes as well as biological measures while preserving the
invariant structure for representing patients' latent mental states. A
multi-layer neural network is used to allow complex treatment effect
heterogeneity. Optimal treatment policy can be inferred for future patients by
comparing their potential mental states under different treatments given the
observed multi-domain pre-treatment measurements. Experiments on simulated data
and a real-world clinical trial data show that the learned treatment polices
compare favorably to alternative methods on heterogeneous treatment effects,
and have broad utilities which lead to better patient outcomes on multiple
domains.

negative:
Adverse drug events (ADEs) are a serious health problem that can be
life-threatening. While a lot of studies have been performed on detect
correlation between a drug and an AE, limited studies have been conducted on
personalized ADE risk prediction. Among treatment alternatives, avoiding the
drug that has high likelihood of causing severe AE can help physicians to
provide safer treatment to patients. Existing work on personalized ADE risk
prediction uses the information obtained in the current medical visit. However,
on the other hand, medical history reveals each patient's unique
characteristics and comprehensive medical information. The goal of this study
is to assess personalized ADE risks that a target drug may induce on a target
patient, based on patient medical history recorded in claims codes, which
provide information about diagnosis, drugs taken, related medical supplies
besides billing information. We developed a HTNNR model (Hierarchical
Time-aware Neural Network for ADE Risk) that capture characteristics of claim
codes and their relationship. The empirical evaluation show that the proposed
HTNNR model substantially outperforms the comparison methods, especially for
rare drugs.

negative:
The huge wealth of data in the health domain can be exploited to create
models that predict development of health states over time. Temporal learning
algorithms are well suited to learn relationships between health states and
make predictions about their future developments. However, these algorithms:
(1) either focus on learning one generic model for all patients, providing
general insights but often with limited predictive performance, or (2) learn
individualized models from which it is hard to derive generic concepts. In this
paper, we present a middle ground, namely parameterized dynamical systems
models that are generated from data using a Genetic Programming (GP) framework.
A fitness function suitable for the health domain is exploited. An evaluation
of the approach in the mental health domain shows that performance of the model
generated by the GP is on par with a dynamical systems model developed based on
domain knowledge, significantly outperforms a generic Long Term Short Term
Memory (LSTM) model and in some cases also outperforms an individualized LSTM
model.

negative:
Driven by the need to capture users' evolving interests and optimize their
long-term experiences, more and more recommender systems have started to model
recommendation as a Markov decision process and employ reinforcement learning
to address the problem. Shouldn't research on the fairness of recommender
systems follow the same trend from static evaluation and one-shot intervention
to dynamic monitoring and non-stop control? In this paper, we portray the
recent developments in recommender systems first and then discuss how fairness
could be baked into the reinforcement learning techniques for recommendation.
Moreover, we argue that in order to make further progress in recommendation
fairness, we may want to consider multi-agent (game-theoretic) optimization,
multi-objective (Pareto) optimization, and simulation-based optimization, in
the general framework of stochastic games.

negative:
Accurate and explainable health event predictions are becoming crucial for
healthcare providers to develop care plans for patients. The availability of
electronic health records (EHR) has enabled machine learning advances in
providing these predictions. However, many deep learning based methods are not
satisfactory in solving several key challenges: 1) effectively utilizing
disease domain knowledge; 2) collaboratively learning representations of
patients and diseases; and 3) incorporating unstructured text. To address these
issues, we propose a collaborative graph learning model to explore
patient-disease interactions and medical domain knowledge. Our solution is able
to capture structural features of both patients and diseases. The proposed
model also utilizes unstructured text data by employing an attention regulation
strategy and then integrates attentive text features into a sequential learning
process. We conduct extensive experiments on two important healthcare problems
to show the competitive prediction performance of the proposed method compared
with various state-of-the-art models. We also confirm the effectiveness of
learned representations and model interpretability by a set of ablation and
case studies.

negative:
Patient similarity assessment, which identifies patients similar to a given
patient, can help improve medical care. The assessment can be performed using
Electronic Medical Records (EMRs). Patient similarity measurement requires
converting heterogeneous EMRs into comparable formats to calculate their
distance. While versatile document representation learning methods have been
developed in recent years, it is still unclear how complex EMR data should be
processed to create the most useful patient representations. This study
presents a new data representation method for EMRs that takes the information
in clinical narratives into account. To address the limitations of previous
approaches in handling complex parts of EMR data, an unsupervised method is
proposed for building a patient representation, which integrates unstructured
data with structured data extracted from patients' EMRs. In order to model the
extracted data, we employed a tree structure that captures the temporal
relations of multiple medical events from EMR. We processed clinical notes to
extract symptoms, signs, and diseases using different tools such as medspaCy,
MetaMap, and scispaCy and mapped entities to the Unified Medical Language
System (UMLS). After creating a tree data structure, we utilized two novel
relabeling methods for the non-leaf nodes of the tree to capture two temporal
aspects of the extracted events. By traversing the tree, we generated a
sequence that could create an embedding vector for each patient. The
comprehensive evaluation of the proposed method for patient similarity and
mortality prediction tasks demonstrated that our proposed model leads to lower
mean squared error (MSE), higher precision, and normalized discounted
cumulative gain (NDCG) relative to baselines.

negative:
As part of a plan launched by the Ministry of Health of Brazil to increase
the availability of linear accelerators for radiotherapy treatment for the
whole country, for which Varian Medical Systems company has won the bidding, a
technical cooperation agreement was signed inviting Brazilian Scientific and
Technological Institutions to participate in a technology transfer program. As
a result, jointly, the Eldorado Research Institute and the Center for
Biomedical Engineering of the University of Campinas presents in this work, the
concepts behind of a proposed rule engine to aid in the evaluation and
decision-making in radiotherapy treatment planning. Normally, the determination
of the radiation dose for a given patient is a complex and intensive procedure,
which requires a lot of domain knowledge and subjective experience from the
oncologists' team. In order to help them in this complex task, and
additionally, provide an auxiliary tool for less experienced oncologists, it is
presented a project conception of a software system that will make use of a
hybrid data-oriented approach. The proposed rule engine will apply both
inference mechanism and expression evaluation to verify and accredit the
quality of an external beam radiation treatment plan by considering, at first,
the 3D-conformal radiotherapy (3DCRT) technique.

negative:
Accurate and transparent prediction of cancer survival times on the level of
individual patients can inform and improve patient care and treatment
practices. In this paper, we design a model that concurrently learns to
accurately predict patient-specific survival distributions and to explain its
predictions in terms of patient attributes such as clinical tests or
assessments. Our model is flexible and based on a recurrent network, can handle
various modalities of data including temporal measurements, and yet constructs
and uses simple explanations in the form of patient- and time-specific linear
regression. For analysis, we use two publicly available datasets and show that
our networks outperform a number of baselines in prediction while providing a
way to inspect the reasons behind each prediction.

negative:
How to leverage cross-document interactions to improve ranking performance is
an important topic in information retrieval (IR) research. However, this topic
has not been well-studied in the learning-to-rank setting and most of the
existing work still treats each document independently while scoring. The
recent development of deep learning shows strength in modeling complex
relationships across sequences and sets. It thus motivates us to study how to
leverage cross-document interactions for learning-to-rank in the deep learning
framework. In this paper, we formally define the permutation-equivariance
requirement for a scoring function that captures cross-document interactions.
We then propose a self-attention based document interaction network and show
that it satisfies the permutation-equivariant requirement, and can generate
scores for document sets of varying sizes. Our proposed methods can
automatically learn to capture document interactions without any auxiliary
information, and can scale across large document sets. We conduct experiments
on three ranking datasets: the benchmark Web30k, a Gmail search, and a Google
Drive Quick Access dataset. Experimental results show that our proposed methods
are both more effective and efficient than baselines.

negative:
Machine learning is used in medicine to support physicians in examination,
diagnosis, and predicting outcomes. One of the most dynamic area is the usage
of patient generated health data from intensive care units. The goal of this
paper is to demonstrate how we advance cross-patient ML model development by
combining the patient's demographics data with their physiological data. We
used a population of patients undergoing Carotid Enderarterectomy (CEA), where
we studied differences in model performance and explainability when trained for
all patients and one patient at a time. The results show that patients'
demographics has a large impact on the performance and explainability and thus
trustworthiness. We conclude that we can increase trust in ML models in a
cross-patient context, by careful selection of models and patients based on
their demographics and the surgical procedure.

negative:
In this paper we describe COLIN, a forward-chaining heuristic search planner,
capable of reasoning with COntinuous LINear numeric change, in addition to the
full temporal semantics of PDDL. Through this work we make two advances to the
state-of-the-art in terms of expressive reasoning capabilities of planners: the
handling of continuous linear change, and the handling of duration-dependent
effects in combination with duration inequalities, both of which require
tightly coupled temporal and numeric reasoning during planning. COLIN combines
FF-style forward chaining search, with the use of a Linear Program (LP) to
check the consistency of the interacting temporal and numeric constraints at
each state. The LP is used to compute bounds on the values of variables in each
state, reducing the range of actions that need to be considered for
application. In addition, we develop an extension of the Temporal Relaxed
Planning Graph heuristic of CRIKEY3, to support reasoning directly with
continuous change. We extend the range of task variables considered to be
suitable candidates for specifying the gradient of the continuous numeric
change effected by an action. Finally, we explore the potential for employing
mixed integer programming as a tool for optimising the timestamps of the
actions in the plan, once a solution has been found. To support this, we
further contribute a selection of extended benchmark domains that include
continuous numeric effects. We present results for COLIN that demonstrate its
scalability on a range of benchmarks, and compare to existing state-of-the-art
planners.

negative:
With increasing and extensive use of electronic health records, clinicians
are often under time pressure when they need to retrieve important information
efficiently among large amounts of patients' health records in clinics. While a
search function can be a useful alternative to browsing through a patient's
record, it is cumbersome for clinicians to search repeatedly for the same or
similar information on similar patients. Under such circumstances, there is a
critical need to build effective recommender systems that can generate accurate
search term recommendations for clinicians. In this manuscript, we developed a
hybrid collaborative filtering model using patients' encounter and search term
information to recommend the next search terms for clinicians to retrieve
important information fast in clinics. For each patient, the model will
recommend terms that either have high co-occurrence frequencies with his/her
most recent ICD codes or are highly relevant to the most recent search terms on
this patient. We have conducted comprehensive experiments to evaluate the
proposed model, and the experimental results demonstrate that our model can
outperform all the state-of-the-art baseline methods for top-N search term
recommendation on different datasets.

negative:
With the successful adoption of machine learning on electronic health records
(EHRs), numerous computational models have been deployed to address a variety
of clinical problems. However, due to the heterogeneity of EHRs, models trained
on different patient groups suffer from poor generalizability. How to mitigate
domain shifts between the source patient group where the model is built upon
and the target one where the model will be deployed becomes a critical issue.
In this paper, we propose a data augmentation method to facilitate domain
adaptation, which leverages knowledge from the source patient group when
training model on the target one. Specifically, adversarially generated samples
are used during domain adaptation to fill the generalization gap between the
two patient groups. The proposed method is evaluated by a case study on
different predictive modeling tasks on MIMIC-III EHR dataset. Results confirm
the effectiveness of our method and the generality on different tasks.

negative:
Management of chronic diseases such as heart failure (HF) is a major public
health problem. A standard approach to managing chronic diseases by medical
community is to have a committee of experts develop guidelines that all
physicians should follow. Due to their complexity, these guidelines are
difficult to implement and are adopted slowly by the medical community at
large. We have developed a physician advisory system that codes the entire set
of clinical practice guidelines for managing HF using answer set
programming(ASP). In this paper we show how abductive reasoning can be deployed
to find missing symptoms and conditions that the patient must exhibit in order
for a treatment prescribed by a physician to work effectively. Thus, if a
physician does not make an appropriate recommendation or makes a non-adherent
recommendation, our system will advise the physician about symptoms and
conditions that must be in effect for that recommendation to apply. It is under
consideration for acceptance in TPLP.

negative:
In clinical practice, physicians make a series of treatment decisions over
the course of a patient's disease based on his/her baseline and evolving
characteristics. A dynamic treatment regime is a set of sequential decision
rules that operationalizes this process. Each rule corresponds to a decision
point and dictates the next treatment action based on the accrued information.
Using existing data, a key goal is estimating the optimal regime, that, if
followed by the patient population, would yield the most favorable outcome on
average. Q- and A-learning are two main approaches for this purpose. We provide
a detailed account of these methods, study their performance, and illustrate
them using data from a depression study.

negative:
Medical AI has tremendous potential to advance healthcare by supporting the
evidence-based practice of medicine, personalizing patient treatment, reducing
costs, and improving provider and patient experience. We argue that unlocking
this potential requires a systematic way to measure the performance of medical
AI models on large-scale heterogeneous data. To meet this need, we are building
MedPerf, an open framework for benchmarking machine learning in the medical
domain. MedPerf will enable federated evaluation in which models are securely
distributed to different facilities for evaluation, thereby empowering
healthcare organizations to assess and verify the performance of AI models in
an efficient and human-supervised process, while prioritizing privacy. We
describe the current challenges healthcare and AI communities face, the need
for an open platform, the design philosophy of MedPerf, its current
implementation status, and our roadmap. We call for researchers and
organizations to join us in creating the MedPerf open benchmarking platform.

negative:
A main research goal in various studies is to use an observational data set
and provide a new set of counterfactual guidelines that can yield causal
improvements. Dynamic Treatment Regimes (DTRs) are widely studied to formalize
this process. However, available methods in finding optimal DTRs often rely on
assumptions that are violated in real-world applications (e.g., medical
decision-making or public policy), especially when (a) the existence of
unobserved confounders cannot be ignored, and (b) the unobserved confounders
are time-varying (e.g., affected by previous actions). When such assumptions
are violated, one often faces ambiguity regarding the underlying causal model.
This ambiguity is inevitable, since the dynamics of unobserved confounders and
their causal impact on the observed part of the data cannot be understood from
the observed data. Motivated by a case study of finding superior treatment
regimes for patients who underwent transplantation in our partner hospital and
faced a medical condition known as New Onset Diabetes After Transplantation
(NODAT), we extend DTRs to a new class termed Ambiguous Dynamic Treatment
Regimes (ADTRs), in which the causal impact of treatment regimes is evaluated
based on a "cloud" of causal models. We then connect ADTRs to Ambiguous
Partially Observable Mark Decision Processes (APOMDPs) and develop
Reinforcement Learning methods, which enable using the observed data to
efficiently learn an optimal treatment regime. We establish theoretical results
for these learning methods, including (weak) consistency and asymptotic
normality. We further evaluate the performance of these learning methods both
in our case study and in simulation experiments.

negative:
We propose a general formulation for continuous treatment recommendation
problems in settings with clinical survival data, which we call the Deep
Survival Dose Response Function (DeepSDRF). That is, we consider the problem of
learning the conditional average dose response (CADR) function solely from
historical data in which observed factors (confounders) affect both observed
treatment and time-to-event outcomes. The estimated treatment effect from
DeepSDRF enables us to develop recommender algorithms with the correction for
selection bias. We compared two recommender approaches based on random search
and reinforcement learning and found similar performance in terms of patient
outcome. We tested the DeepSDRF and the corresponding recommender on extensive
simulation studies and the eICU Research Institute (eRI) database. To the best
of our knowledge, this is the first time that causal models are used to address
the continuous treatment effect with observational data in a medical context.

negative:
Today, treatment effect estimation at the individual level is a vital problem
in many areas of science and business. For example, in marketing, estimates of
the treatment effect are used to select the most efficient promo-mechanics; in
medicine, individual treatment effects are used to determine the optimal dose
of medication for each patient and so on. At the same time, the question on
choosing the best method, i.e., the method that ensures the smallest predictive
error (for instance, RMSE) or the highest total (average) value of the effect,
remains open. Accordingly, in this paper we compare the effectiveness of
machine learning methods for estimation of individual treatment effects. The
comparison is performed on the Criteo Uplift Modeling Dataset. In this paper we
show that the combination of the Logistic Regression method and the Difference
Score method as well as Uplift Random Forest method provide the best
correctness of Individual Treatment Effect prediction on the top 30\%
observations of the test dataset.

negative:
Temporal planning often involves numeric effects that are directly
proportional to their action's duration. These include continuous effects,
where a numeric variable is subjected to a rate of change while the action is
being executed, and discrete duration-dependent effects, where the variable is
updated instantaneously but the magnitude of such change is computed from the
action's duration. When these effects are linear, state--of--the--art temporal
planners often make use of Linear Programming to ensure that these numeric
updates are consistent with the chosen start times and durations of the plan's
actions. This is typically done for each evaluated state as part of the search
process. This exhaustive approach is not scalable to solve real-world problems
that require long plans, because the linear program's size becomes larger and
slower to solve. In this work we propose techniques that minimise this overhead
by computing these checks more selectively and formulating linear programs that
have a smaller footprint. The effectiveness of these techniques is demonstrated
on domains that use a mix of discrete and continuous effects, which is typical
of real-world planning problems. The resultant planner also outperforms most
state-of-the-art temporal-numeric and hybrid planners, in terms of both
coverage and scalability.

negative:
Clinical trials play important roles in drug development but often suffer
from expensive, inaccurate and insufficient patient recruitment. The
availability of massive electronic health records (EHR) data and trial
eligibility criteria (EC) bring a new opportunity to data driven patient
recruitment. One key task named patient-trial matching is to find qualified
patients for clinical trials given structured EHR and unstructured EC text
(both inclusion and exclusion criteria). How to match complex EC text with
longitudinal patient EHRs? How to embed many-to-many relationships between
patients and trials? How to explicitly handle the difference between inclusion
and exclusion criteria? In this paper, we proposed CrOss-Modal PseudO-SiamEse
network (COMPOSE) to address these challenges for patient-trial matching. One
path of the network encodes EC using convolutional highway network. The other
path processes EHR with multi-granularity memory network that encodes
structured patient records into multiple levels based on medical ontology.
Using the EC embedding as query, COMPOSE performs attentional record alignment
and thus enables dynamic patient-trial matching. COMPOSE also introduces a
composite loss term to maximize the similarity between patient records and
inclusion criteria while minimize the similarity to the exclusion criteria.
Experiment results show COMPOSE can reach 98.0% AUC on patient-criteria
matching and 83.7% accuracy on patient-trial matching, which leads 24.3%
improvement over the best baseline on real-world patient-trial matching tasks.

negative:
In realistic scenarios, multivariate timeseries evolve over case-by-case
time-scales. This is particularly clear in medicine, where the rate of clinical
events varies by ward, patient, and application. Increasingly complex models
have been shown to effectively predict patient outcomes, but have failed to
adapt granularity to these inherent temporal resolutions. As such, we introduce
a novel, more realistic, approach to generating patient outcome predictions at
an adaptive rate based on uncertainty accumulation in Bayesian recurrent
models. We use a Recurrent Neural Network (RNN) and a Bayesian embedding layer
with a new aggregation method to demonstrate adaptive prediction timing. Our
model predicts more frequently when events are dense or the model is certain of
event latent representations, and less frequently when readings are sparse or
the model is uncertain. At 48 hours after patient admission, our model achieves
equal performance compared to its static-windowed counterparts, while
generating patient- and event-specific prediction timings that lead to improved
predictive performance over the crucial first 12 hours of the patient stay.

negative:
Sepsis is the leading cause of mortality in the ICU. It is challenging to
manage because individual patients respond differently to treatment. Thus,
tailoring treatment to the individual patient is essential for the best
outcomes. In this paper, we take steps toward this goal by applying a
mixture-of-experts framework to personalize sepsis treatment. The mixture model
selectively alternates between neighbor-based (kernel) and deep reinforcement
learning (DRL) experts depending on patient's current history. On a large
retrospective cohort, this mixture-based approach outperforms physician, kernel
only, and DRL-only experts.

negative:
When observational data is available from practical studies and a directed
cyclic graph for how various variables affect each other is known based on
substantive understanding of the process, we consider a problem in which a
control plan of a treatment variable is conducted in order to bring a response
variable close to a target value with variation reduction. We formulate an
optimal control plan concerning a certain treatment variable through path
coefficients in the framework of linear nonrecursive structural equation
models. Based on the formulation, we clarify the properties of causal effects
when conducting a control plan. The results enable us to evaluate the effect of
a control plan on the variance from observational data.

negative:
Given recent deep learning results that demonstrate the ability to
effectively optimize high-dimensional non-convex functions with gradient
descent optimization on GPUs, we ask in this paper whether symbolic gradient
optimization tools such as Tensorflow can be effective for planning in hybrid
(mixed discrete and continuous) nonlinear domains with high dimensional state
and action spaces? To this end, we demonstrate that hybrid planning with
Tensorflow and RMSProp gradient descent is competitive with mixed integer
linear program (MILP) based optimization on piecewise linear planning domains
(where we can compute optimal solutions) and substantially outperforms
state-of-the-art interior point methods for nonlinear planning domains.
Furthermore, we remark that Tensorflow is highly scalable, converging to a
strong plan on a large-scale concurrent domain with a total of 576,000
continuous action parameters distributed over a horizon of 96 time steps and
100 parallel instances in only 4 minutes. We provide a number of insights that
clarify such strong performance including observations that despite long
horizons, RMSProp avoids both the vanishing and exploding gradient problems.
Together these results suggest a new frontier for highly scalable planning in
nonlinear hybrid domains by leveraging GPUs and the power of recent advances in
gradient descent with highly optimized toolkits like Tensorflow.

negative:
Personalized medicine, a paradigm of medicine tailored to a patient's
characteristics, is an increasingly attractive field in health care. An
important goal of personalized medicine is to identify a subgroup of patients,
based on baseline covariates, that benefits more from the targeted treatment
than other comparative treatments. Most of the current subgroup identification
methods only focus on obtaining a subgroup with an enhanced treatment effect
without paying attention to subgroup size. Yet, a clinically meaningful
subgroup learning approach should identify the maximum number of patients who
can benefit from the better treatment. In this paper, we present an optimal
subgroup selection rule (SSR) that maximizes the number of selected patients,
and in the meantime, achieves the pre-specified clinically meaningful mean
outcome, such as the average treatment effect. We derive two equivalent
theoretical forms of the optimal SSR based on the contrast function that
describes the treatment-covariates interaction in the outcome. We further
propose a ConstrAined PolIcy Tree seArch aLgorithm (CAPITAL) to find the
optimal SSR within the interpretable decision tree class. The proposed method
is flexible to handle multiple constraints that penalize the inclusion of
patients with negative treatment effects, and to address time to event data
using the restricted mean survival time as the clinically interesting mean
outcome. Extensive simulations, comparison studies, and real data applications
are conducted to demonstrate the validity and utility of our method.

negative:
Widespread adoption of electronic health records (EHRs) has fueled the
development of using machine learning to build prediction models for various
clinical outcomes. This process is often constrained by having a relatively
small number of patient records for training the model. We demonstrate that
using patient representation schemes inspired from techniques in natural
language processing can increase the accuracy of clinical prediction models by
transferring information learned from the entire patient population to the task
of training a specific model, where only a subset of the population is
relevant. Such patient representation schemes enable a 3.5% mean improvement in
AUROC on five prediction tasks compared to standard baselines, with the average
improvement rising to 19% when only a small number of patient records are
available for training the clinical prediction model.

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
The introduction of data analytics into medicine has changed the nature of
patient treatment. In this, patients are asked to disclose personal information
such as genetic markers, lifestyle habits, and clinical history. This data is
then used by statistical models to predict personalized treatments. However,
due to privacy concerns, patients often desire to withhold sensitive
information. This self-censorship can impede proper diagnosis and treatment,
which may lead to serious health complications and even death over time. In
this paper, we present privacy distillation, a mechanism which allows patients
to control the type and amount of information they wish to disclose to the
healthcare providers for use in statistical models. Meanwhile, it retains the
accuracy of models that have access to all patient data under a sufficient but
not full set of privacy-relevant information. We validate privacy distillation
using a corpus of patients prescribed to warfarin for a personalized dosage. We
use a deep neural network to implement privacy distillation for training and
making dose predictions. We find that privacy distillation with sufficient
privacy-relevant information i) retains accuracy almost as good as having all
patient data (only 3\% worse), and ii) is effective at preventing errors that
introduce health-related risks (only 3.9\% worse under- or over-prescriptions).

negative:
Tens of millions of women suffer from infertility worldwide each year. In
vitro fertilization (IVF) is the best choice for many such patients. However,
IVF is expensive, time-consuming, and both physically and emotionally
demanding. The first question that a patient usually asks before the IVF is how
likely she will conceive, given her basic medical examination information. This
paper proposes three approaches to predict the cumulative pregnancy rate after
multiple oocyte pickup cycles. Experiments on 11,190 patients showed that first
clustering the patients into different groups and then building a support
vector machine model for each group can achieve the best overall performance.
Our model could be a quick and economic approach for reliably estimating the
cumulative pregnancy rate for a patient, given only her basic medical
examination information, well before starting the actual IVF procedure. The
predictions can help the patient make optimal decisions on whether to use her
own oocyte or donor oocyte, how many oocyte pickup cycles she may need, whether
to use embryo frozen, etc. They will also reduce the patient's cost and time to
pregnancy, and improve her quality of life.

negative:
Patient similarity analysis is important in health care applications. It
takes patient information such as their electronic medical records and genetic
data as input and computes the pairwise similarity between patients. Procedures
of typical a patient similarity study can be divided into several steps
including data integration, similarity measurement, and neighborhood
identification. And according to an analysis of patient similarity, doctors can
easily find the most suitable treatments. There are many methods to analyze the
similarity such as cluster analysis. And during machine learning become more
and more popular, Using neural networks such as CNN is a new hot topic. This
review summarizes representative methods used in each step and discusses
applications of patient similarity networks especially in the context of
precision medicine.

negative:
With the increase of information, document classification as one of the
methods of text mining, plays vital role in many management and organizing
information. Document classification is the process of assigning a document to
one or more predefined category labels. Document classification includes
different parts such as text processing, term selection, term weighting and
final classification. The accuracy of document classification is very
important. Thus improvement in each part of classification should lead to
better results and higher precision. Term weighting has a great impact on the
accuracy of the classification. Most of the existing weighting methods exploit
the statistical information of terms in documents and do not consider semantic
relations between words. In this paper, an automated document classification
system is presented that uses a novel term weighting method based on semantic
relations between terms. To evaluate the proposed method, three standard
Persian corpuses are used. Experiment results show 2 to 4 percent improvement
in classification accuracy compared with the best previous designed system for
Persian documents.

negative:
Alzheimer's disease (AD) is a degenerative brain disease impairing a person's
ability to perform day to day activities. The clinical manifestations of
Alzheimer's disease are characterized by heterogeneity in age, disease span,
progression rate, impairment of memory and cognitive abilities. Due to these
variabilities, personalized care and treatment planning, as well as patient
counseling about their individual progression is limited. Recent developments
in machine learning to detect hidden patterns in complex, multi-dimensional
datasets provides significant opportunities to address this critical need. In
this work, we use unsupervised and supervised machine learning approaches for
subtype identification and prediction. We apply machine learning methods to the
extensive clinical observations available at the Alzheimer's Disease
Neuroimaging Initiative (ADNI) data set to identify patient subtypes and to
predict disease progression. Our analysis depicts the progression space for the
Alzheimer's disease into low, moderate and high disease progression zones. The
proposed work will enable early detection and characterization of distinct
disease subtypes based on clinical heterogeneity. We anticipate that our models
will enable patient counseling, clinical trial design, and ultimately
individualized clinical care.

negative:
Avoiding collisions is one of the vital tasks for systems of autonomous
mobile agents. We focus on the problem of finding continuous coordinated paths
for multiple mobile disc agents in a 2-d environment with polygonal obstacles.
The problem is PSPACE-hard, with the state space growing exponentially in the
number of agents. Therefore, the state of the art methods include mainly
reactive techniques and sampling-based iterative algorithms.
  We compare the performance of a widely-used reactive method ORCA with three
variants of a popular planning algorithm RRT* applied to multi-agent path
planning and find that an algorithm combining reactive collision avoidance and
RRT* planning, which we call ORCA-RRT* can be used to solve instances that are
out of the reach of either of the techniques. We experimentally show that: 1)
the reactive part of the algorithm can efficiently solve many multi-agent path
finding problems involving large number of agents, for which RRT* algorithm is
often unable to find a solution in limited time and 2) the planning component
of the algorithm is able to solve many instances containing local minima, where
reactive techniques typically fail.

negative:
Traditional medicine typically applies one-size-fits-all treatment for the
entire patient population whereas precision medicine develops tailored
treatment schemes for different patient subgroups. The fact that some factors
may be more significant for a specific patient subgroup motivates clinicians
and medical researchers to develop new approaches to subgroup detection and
analysis, which is an effective strategy to personalize treatment. In this
study, we propose a novel patient subgroup detection method, called Supervised
Biclustring (SUBIC) using convex optimization and apply our approach to detect
patient subgroups and prioritize risk factors for hypertension (HTN) in a
vulnerable demographic subgroup (African-American). Our approach not only finds
patient subgroups with guidance of a clinically relevant target variable but
also identifies and prioritizes risk factors by pursuing sparsity of the input
variables and encouraging similarity among the input variables and between the
input and target variables

negative:
Current AI approaches have frequently been used to help personalize many
aspects of medical experiences and tailor them to a specific individuals'
needs. However, while such systems consider medically-relevant information,
they ignore socially-relevant information about how this diagnosis should be
communicated and discussed with the patient. The lack of this capability may
lead to mis-communication, resulting in serious implications, such as patients
opting out of the best treatment. Consider a case in which the same treatment
is proposed to two different individuals. The manner in which this treatment is
mediated to each should be different, depending on the individual patient's
history, knowledge, and mental state. While it is clear that this communication
should be conveyed via a human medical expert and not a software-based system,
humans are not always capable of considering all of the relevant aspects and
traversing all available information. We pose the challenge of creating
Intelligent Agents (IAs) to assist medical service providers (MSPs) and
consumers in establishing a more personalized human-to-human dialogue.
Personalizing conversations will enable patients and MSPs to reach a solution
that is best for their particular situation, such that a relation of trust can
be built and commitment to the outcome of the interaction is assured. We
propose a four-part conceptual framework for personalized social interactions,
expand on which techniques are available within current AI research and discuss
what has yet to be achieved.

negative:
Recent approaches in literature have exploited the multi-modal information in
documents (text, layout, image) to serve specific downstream document tasks.
However, they are limited by their - (i) inability to learn cross-modal
representations across text, layout and image dimensions for documents and (ii)
inability to process multi-page documents. Pre-training techniques have been
shown in Natural Language Processing (NLP) domain to learn generic textual
representations from large unlabelled datasets, applicable to various
downstream NLP tasks. In this paper, we propose a multi-task learning-based
framework that utilizes a combination of self-supervised and supervised
pre-training tasks to learn a generic document representation applicable to
various downstream document tasks. Specifically, we introduce Document Topic
Modelling and Document Shuffle Prediction as novel pre-training tasks to learn
rich image representations along with the text and layout representations for
documents. We utilize the Longformer network architecture as the backbone to
encode the multi-modal information from multi-page documents in an end-to-end
fashion. We showcase the applicability of our pre-training framework on a
variety of different real-world document tasks such as document classification,
document information extraction, and document retrieval. We evaluate our
framework on different standard document datasets and conduct exhaustive
experiments to compare performance against various ablations of our framework
and state-of-the-art baselines.

negative:
Electronic health records (EHRs) are longitudinal records of a patient's
interactions with healthcare systems. A patient's EHR data is organized as a
three-level hierarchy from top to bottom: patient journey - all the experiences
of diagnoses and treatments over a period of time; individual visit - a set of
medical codes in a particular visit; and medical code - a specific record in
the form of medical codes. As EHRs begin to amass in millions, the potential
benefits, which these data might hold for medical research and medical outcome
prediction, are staggering - including, for example, predicting future
admissions to hospitals, diagnosing illnesses or determining the efficacy of
medical treatments. Each of these analytics tasks requires a domain knowledge
extraction method to transform the hierarchical patient journey into a vector
representation for further prediction procedure. The representations should
embed a sequence of visits and a set of medical codes with a specific
timestamp, which are crucial to any downstream prediction tasks. Hence,
expressively powerful representations are appealing to boost learning
performance. To this end, we propose a novel self-attention mechanism that
captures the contextual dependency and temporal relationships within a
patient's healthcare journey. An end-to-end bidirectional temporal encoder
network (BiteNet) then learns representations of the patient's journeys, based
solely on the proposed attention mechanism. We have evaluated the effectiveness
of our methods on two supervised prediction and two unsupervised clustering
tasks with a real-world EHR dataset. The empirical results demonstrate the
proposed BiteNet model produces higher-quality representations than
state-of-the-art baseline methods.

negative:
In several medical decision-making problems, such as antibiotic prescription,
laboratory testing can provide precise indications for how a patient will
respond to different treatment options. This enables us to "fully observe" all
potential treatment outcomes, but while present in historical data, these
results are infeasible to produce in real-time at the point of the initial
treatment decision. Moreover, treatment policies in these settings often need
to trade off between multiple competing objectives, such as effectiveness of
treatment and harmful side effects. We present, compare, and evaluate three
approaches for learning individualized treatment policies in this setting:
First, we consider two indirect approaches, which use predictive models of
treatment response to construct policies optimal for different trade-offs
between objectives. Second, we consider a direct approach that constructs such
a set of policies without intermediate models of outcomes. Using a medical
dataset of Urinary Tract Infection (UTI) patients, we show that all approaches
learn policies that achieve strictly better performance on all outcomes than
clinicians, while also trading off between different objectives. We demonstrate
additional benefits of the direct approach, including flexibly incorporating
other goals such as deferral to physicians on simple cases.

negative:
This paper presents a probabilistic model for reasoning about the state of a
system as it changes over time, both due to exogenous and endogenous
influences. Our target domain is a class of medical prediction problems that
are neither so urgent as to preclude careful diagnosis nor progress so slowly
as to allow arbitrary testing and treatment options. In these domains there is
typically enough time to gather information about the patient's state and
consider alternative diagnoses and treatments, but the temporal interaction
between the timing of tests, treatments, and the course of the disease must
also be considered. Our approach is to elicit a qualitative structural model of
the patient from a human expert---the model identifies important attributes,
the way in which exogenous changes affect attribute values, and the way in
which the patient's condition changes endogenously. We then elicit
probabilistic information to capture the expert's uncertainty about the effects
of tests and treatments and the nature and timing of endogenous state changes.
This paper describes the model in the context of a problem in treating vehicle
accident trauma, and suggests a method for solving the model based on the
technique of sequential imputation. A complementary goal of this work is to
understand and synthesize a disparate collection of research efforts all using
the name ?probabilistic temporal reasoning.? This paper analyzes related work
and points out essential differences between our proposed model and other
approaches in the literature.

negative:
Clinical event sequences consist of thousands of clinical events that
represent records of patient care in time. Developing accurate prediction
models for such sequences is of a great importance for defining representations
of a patient state and for improving patient care. One important challenge of
learning a good predictive model of clinical sequences is patient-specific
variability. Based on underlying clinical complications, each patient's
sequence may consist of different sets of clinical events. However,
population-based models learned from such sequences may not accurately predict
patient-specific dynamics of event sequences. To address the problem, we
develop a new adaptive event sequence prediction framework that learns to
adjust its prediction for individual patients through an online model update.

negative:
A crucial step within secondary analysis of electronic health records (EHRs)
is to identify the patient cohort under investigation. While EHRs contain
medical billing codes that aim to represent the conditions and treatments
patients may have, much of the information is only present in the patient
notes. Therefore, it is critical to develop robust algorithms to infer
patients' conditions and treatments from their written notes. In this paper, we
introduce a dataset for patient phenotyping, a task that is defined as the
identification of whether a patient has a given medical condition (also
referred to as clinical indication or phenotype) based on their patient note.
Nursing Progress Notes and Discharge Summaries from the Intensive Care Unit of
a large tertiary care hospital were manually annotated for the presence of
several high-context phenotypes relevant to treatment and risk of
re-hospitalization. This dataset contains 1102 Discharge Summaries and 1000
Nursing Progress Notes. Each Discharge Summary and Progress Note has been
annotated by at least two expert human annotators (one clinical researcher and
one resident physician). Annotated phenotypes include treatment non-adherence,
chronic pain, advanced/metastatic cancer, as well as 10 other phenotypes. This
dataset can be utilized for academic and industrial research in medicine and
computer science, particularly within the field of medical natural language
processing.

negative:
Contemporary patient surveillance systems have streamlined central
surveillance into the electronic health record interface. They are able to
process the sheer volume of patient data by adopting machine learning
approaches. However, these systems are not suitable for implementation in many
hospitals, mostly in developing countries, with limited human, financial, and
technological resources. Through conducting thorough research on intensive care
facilities, we designed a novel central patient monitoring system and in this
paper, we describe the working prototype of our system. The proposed prototype
comprises of inexpensive peripherals and simplistic user interface. Our central
patient monitoring system implements Kernel-based On-line Anomaly Detection
(KOAD) algorithm for emergency event signaling. By evaluating continuous
patient data, we show that the system is able to detect critical events in
real-time reliably and has low false alarm rate.

negative:
The quality of a document is affected by various factors, including
grammaticality, readability, stylistics, and expertise depth, making the task
of document quality assessment a complex one. In this paper, we explore this
task in the context of assessing the quality of Wikipedia articles and academic
papers. Observing that the visual rendering of a document can capture implicit
quality indicators that are not present in the document text --- such as
images, font choices, and visual layout --- we propose a joint model that
combines the text content with a visual rendering of the document for document
quality assessment. Experimental results over two datasets reveal that textual
and visual features are complementary, achieving state-of-the-art results.

negative:
The main question to address in this paper is to recommend optimal signal
timing plans in real time under incidents by incorporating domain knowledge
developed with the traffic signal timing plans tuned for possible incidents,
and learning from historical data of both traffic and implemented signals
timing. The effectiveness of traffic incident management is often limited by
the late response time and excessive workload of traffic operators. This paper
proposes a novel decision-making framework that learns from both data and
domain knowledge to real-time recommend contingency signal plans that
accommodate non-recurrent traffic, with the outputs from real-time traffic
prediction at least 30 minutes in advance. Specifically, considering the rare
occurrences of engagement of contingency signal plans for incidents, we propose
to decompose the end-to-end recommendation task into two hierarchical models:
real-time traffic prediction and plan association. We learn the connections
between the two models through metric learning, which reinforces partial-order
preferences observed from historical signal engagement records. We demonstrate
the effectiveness of our approach by testing this framework on the traffic
network in Cranberry Township in 2019. Results show that our recommendation
system has a precision score of 96.75% and recall of 87.5% on the testing plan,
and make recommendation of an average of 22.5 minutes lead time ahead of Waze
alerts. The results suggest that our framework is capable of giving traffic
operators a significant time window to access the conditions and respond
appropriately.

negative:
More than 144 000 Australians were diagnosed with cancer in 2019. The
majority will first present to their GP symptomatically, even for cancer for
which screening programs exist. Diagnosing cancer in primary care is
challenging due to the non-specific nature of cancer symptoms and its low
prevalence. Understanding the epidemiology of cancer symptoms and patterns of
presentation in patient's medical history from primary care data could be
important to improve earlier detection and cancer outcomes. As past medical
data about a patient can be incomplete, irregular or missing, this creates
additional challenges when attempting to use the patient's history for any new
diagnosis. Our research aims to investigate the opportunities in a patient's
pathology history available to a GP, initially focused on the results within
the frequently ordered full blood count to determine relevance to a future
high-risk cancer prognosis, and treatment outcome. We investigated how past
pathology test results can lead to deriving features that can be used to
predict cancer outcomes, with emphasis on patients at risk of not surviving the
cancer within 2-year period. This initial work focuses on patients with lung
cancer, although the methodology can be applied to other types of cancer and
other data within the medical record. Our findings indicate that even in cases
of incomplete or obscure patient history, hematological measures can be useful
in generating features relevant for predicting cancer risk and survival. The
results strongly indicate to add the use of pathology test data for potential
high-risk cancer diagnosis, and the utilize additional pathology metrics or
other primary care datasets even more for similar purposes.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
This paper presents a new state-of-the-art for document image classification
and retrieval, using features learned by deep convolutional neural networks
(CNNs). In object and scene analysis, deep neural nets are capable of learning
a hierarchical chain of abstraction from pixel inputs to concise and
descriptive representations. The current work explores this capacity in the
realm of document analysis, and confirms that this representation strategy is
superior to a variety of popular hand-crafted alternatives. Experiments also
show that (i) features extracted from CNNs are robust to compression, (ii) CNNs
trained on non-document images transfer well to document analysis tasks, and
(iii) enforcing region-specific feature-learning is unnecessary given
sufficient training data. This work also makes available a new labelled subset
of the IIT-CDIP collection, containing 400,000 document images across 16
categories, useful for training new CNNs for document analysis.

negative:
Practitioners in medicine, business, political science, and other fields are
increasingly aware that decisions should be personalized to each patient,
customer, or voter. A given treatment (e.g. a drug or advertisement) should be
administered only to those who will respond most positively, and certainly not
to those who will be harmed by it. Individual-level treatment effects can be
estimated with tools adapted from machine learning, but different models can
yield contradictory estimates. Unlike risk prediction models, however,
treatment effect models cannot be easily evaluated against each other using a
held-out test set because the true treatment effect itself is never directly
observed. Besides outcome prediction accuracy, several metrics that can
leverage held-out data to evaluate treatment effects models have been proposed,
but they are not widely used. We provide a didactic framework that elucidates
the relationships between the different approaches and compare them all using a
variety of simulations of both randomized and observational data. Our results
show that researchers estimating heterogenous treatment effects need not limit
themselves to a single model-fitting algorithm. Instead of relying on a single
method, multiple models fit by a diverse set of algorithms should be evaluated
against each other using an objective function learned from the validation set.
The model minimizing that objective should be used for estimating the
individual treatment effect for future individuals.

negative:
One of the major prerequisites for any deep learning approach is the
availability of large-scale training data. When dealing with scanned document
images in real world scenarios, the principal information of its content is
stored in the layout itself. In this work, we have proposed an automated deep
generative model using Graph Neural Networks (GNNs) to generate synthetic data
with highly variable and plausible document layouts that can be used to train
document interpretation systems, in this case, specially in digital mailroom
applications. It is also the first graph-based approach for document layout
generation task experimented on administrative document images, in this case,
invoices.

negative:
In recent years, people from all over the world are suffering from one of the
most severe diseases in history, known as Coronavirus disease 2019, COVID-19
for short. When the virus reaches the lungs, it has a higher probability to
cause lung pneumonia and sepsis. X-ray image is a powerful tool in identifying
the typical features of the infection for COVID-19 patients. The radiologists
and pathologists observe that ground-glass opacity appears in the chest X-ray
for infected patient \cite{cozzi2021ground}, and it could be used as one of the
criteria during the diagnosis process. In the past few years, deep learning has
proven to be one of the most powerful methods in the field of image
classification. Due to significant differences in Chest X-Ray between normal
and infected people \cite{rousan2020chest}, deep models could be used to
identify the presence of the disease given a patient's Chest X-Ray. Many deep
models are complex, and it evolves with lots of input parameters. Designers
sometimes struggle with the tuning process for deep models, especially when
they build up the model from scratch. Genetic Algorithm, inspired by the
biological evolution process, plays a key role in solving such complex
problems. In this paper, I proposed a genetic-based approach to optimize the
Convolutional Neural Network(CNN) for the Chest X-Ray classification task.

negative:
Critically ill patients in regular wards are vulnerable to unanticipated
adverse events which require prompt transfer to the intensive care unit (ICU).
To allow for accurate prognosis of deteriorating patients, we develop a novel
continuous-time probabilistic model for a monitored patient's temporal sequence
of physiological data. Our model captures "informatively sampled" patient
episodes: the clinicians' decisions on when to observe a hospitalized patient's
vital signs and lab tests over time are represented by a marked Hawkes process,
with intensity parameters that are modulated by the patient's latent clinical
states, and with observable physiological data (mark process) modeled as a
switching multi-task Gaussian process. In addition, our model captures
"informatively censored" patient episodes by representing the patient's latent
clinical states as an absorbing semi-Markov jump process. The model parameters
are learned from offline patient episodes in the electronic health records via
an EM-based algorithm. Experiments conducted on a cohort of patients admitted
to a major medical center over a 3-year period show that risk prognosis based
on our model significantly outperforms the currently deployed medical risk
scores and other baseline machine learning algorithms.

negative:
To address potential gaps noted in patient monitoring in the hospital, a
novel patient behavior detection system using mmWave radar and deep convolution
neural network (CNN), which supports the simultaneous recognition of multiple
patients' behaviors in real-time, is proposed. In this study, we use an mmWave
radar to track multiple patients and detect the scattering point cloud of each
one. For each patient, the Doppler pattern of the point cloud over a time
period is collected as the behavior signature. A three-layer CNN model is
created to classify the behavior for each patient. The tracking and point
clouds detection algorithm was also implemented on an mmWave radar hardware
platform with an embedded graphics processing unit (GPU) board to collect
Doppler pattern and run the CNN model. A training dataset of six types of
behavior were collected, over a long duration, to train the model using Adam
optimizer with an objective to minimize cross-entropy loss function. Lastly,
the system was tested for real-time operation and obtained a very good
inference accuracy when predicting each patient's behavior in a two-patient
scenario.

negative:
One major drawback of state of the art Neural Networks (NN)-based approaches
for document classification purposes is the large number of training samples
required to obtain an efficient classification. The minimum required number is
around one thousand annotated documents for each class. In many cases it is
very difficult, if not impossible, to gather this number of samples in real
industrial processes. In this paper, we analyse the efficiency of NN-based
document classification systems in a sub-optimal training case, based on the
situation of a company document stream. We evaluated three different
approaches, one based on image content and two on textual content. The
evaluation was divided into four parts: a reference case, to assess the
performance of the system in the lab; two cases that each simulate a specific
difficulty linked to document stream processing; and a realistic case that
combined all of these difficulties. The realistic case highlighted the fact
that there is a significant drop in the efficiency of NN-Based document
classification systems. Although they remain efficient for well represented
classes (with an over-fitting of the system for those classes), it is
impossible for them to handle appropriately less well represented classes.
NN-Based document classification systems need to be adapted to resolve these
two problems before they can be considered for use in a company document
stream.

negative:
Document date is essential for many important tasks, such as document
retrieval, summarization, event detection, etc. While existing approaches for
these tasks assume accurate knowledge of the document date, this is not always
available, especially for arbitrary documents from the Web. Document Dating is
a challenging problem which requires inference over the temporal structure of
the document. Prior document dating systems have largely relied on handcrafted
features while ignoring such document internal structures. In this paper, we
propose NeuralDater, a Graph Convolutional Network (GCN) based document dating
approach which jointly exploits syntactic and temporal graph structures of
document in a principled way. To the best of our knowledge, this is the first
application of deep learning for the problem of document dating. Through
extensive experiments on real-world datasets, we find that NeuralDater
significantly outperforms state-of-the-art baseline by 19% absolute (45%
relative) accuracy points.

negative:
Most of the textual information available to us are temporally variable. In a
world where information is dynamic, time-stamping them is a very important
task. Documents are a good source of information and are used for many tasks
like, sentiment analysis, classification of reviews etc. The knowledge of
creation date of documents facilitates several tasks like summarization, event
extraction, temporally focused information extraction etc. Unfortunately, for
most of the documents on the web, the time-stamp meta-data is either erroneous
or missing. Thus document dating is a challenging problem which requires
inference over the temporal structure of the document alongside the contextual
information of the document. Prior document dating systems have largely relied
on handcrafted features while ignoring such document-internal structures. In
this paper we propose NeuralDater, a Graph Convolutional Network (GCN) based
document dating approach which jointly exploits syntactic and temporal graph
structures of document in a principled way. We also pointed out some
limitations of NeuralDater and tried to utilize both context and temporal
information in documents in a more flexible and intuitive manner proposing AD3:
Attentive Deep Document Dater, an attention-based document dating system. To
the best of our knowledge these are the first application of deep learning
methods for the task. Through extensive experiments on real-world datasets, we
find that our models significantly outperforms state-of-the-art baselines by a
significant margin.