query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : I am interested in understanding how modern computer programs can emulate human causal logic. To achieve this, I would like to learn about the latest computational models that can predict human judgments and compare their performance with that of humans. Additionally, I am seeking references where human participants are presented with causal structures generated by Bayesian-related models, along with their explanations or feedback based on these model-generated outputs. I am curious about the most common differences between human decision-making and machine-based decisions, as well as the history of the development of models for causal reasoning, particularly how these models draw inspiration from the neural structures of living creatures.

positive:
Research in Cognitive Science suggests that humans understand and represent
knowledge of the world through causal relationships. In addition to
observations, they can rely on experimenting and counterfactual reasoning --
i.e. referring to an alternative course of events -- to identify causal
relations and explain atypical situations. Different instances of control
systems, such as smart homes, would benefit from having a similar causal model,
as it would help the user understand the logic of the system and better react
when needed. However, while data-driven methods achieve high levels of
correlation detection, they mainly fall short of finding causal relations,
notably being limited to observations only. Notably, they struggle to identify
the cause from the effect when detecting a correlation between two variables.
This paper introduces a new way to learn causal models from a mixture of
experiments on the environment and observational data. The core of our method
is the use of selected interventions, especially our learning takes into
account the variables where it is impossible to intervene, unlike other
approaches. The causal model we obtain is then used to generate Causal Bayesian
Networks, which can be later used to perform diagnostic and predictive
inference. We use our method on a smart home simulation, a use case where
knowing causal relations pave the way towards explainable systems. Our
algorithm succeeds in generating a Causal Bayesian Network close to the
simulation's ground truth causal interactions, showing encouraging prospects
for application in real-life systems.

positive:
A major challenge in research involving artificial intelligence (AI) is the
development of algorithms that can find solutions to problems that can
generalize to different environments and tasks. Unlike AI, humans are adept at
finding solutions that can transfer. We hypothesize this is because their
solutions are informed by causal models. We propose to use human-guided causal
knowledge to help robots find solutions that can generalize to a new
environment. We develop and test the feasibility of a language interface that
na\"ive participants can use to communicate these causal models to a planner.
We find preliminary evidence that participants are able to use our interface
and generate causal models that achieve near-generalization. We outline an
experiment aimed at testing far-generalization using our interface and describe
our longer terms goals for these causal models.

positive:
Discriminating between causality and correlation is a major problem in
machine learning, and theoretical tools for determining causality are still
being developed. However, people commonly make causality judgments and are
often correct, even in unfamiliar domains. What are humans doing to make these
judgments? This paper examines differences in human experts' and non-experts'
ability to attribute causality by comparing their performances to those of
machine-learning algorithms. We collected human judgments by using Amazon
Mechanical Turk (MTurk) and then divided the human subjects into two groups:
experts and non-experts. We also prepared expert and non-expert machine
algorithms based on different training of convolutional neural network (CNN)
models. The results showed that human experts' judgments were similar to those
made by an "expert" CNN model trained on a large number of examples from the
target domain. The human non-experts' judgments resembled the prediction
outputs of the CNN model that was trained on only the small number of examples
used during the MTurk instruction. We also analyzed the differences between the
expert and non-expert machine algorithms based on their neural representations
to evaluate the performances, providing insight into the human experts' and
non-experts' cognitive abilities.

positive:
Causal Models are increasingly suggested as a means to reason about the
behavior of cyber-physical systems in socio-technical contexts. They allow us
to analyze courses of events and reason about possible alternatives. Until now,
however, such reasoning is confined to the technical domain and limited to
single systems or at most groups of systems. The humans that are an integral
part of any such socio-technical system are usually ignored or dealt with by
"expert judgment". We show how a technical causal model can be extended with
models of human behavior to cover the complexity and interplay between humans
and technical systems. This integrated socio-technical causal model can then be
used to reason not only about actions and decisions taken by the machine, but
also about those taken by humans interacting with the system. In this paper we
demonstrate the feasibility of merging causal models about machines with causal
models about humans and illustrate the usefulness of this approach with a
highly automated vehicle example.

positive:
Humans demonstrate remarkable abilities to predict physical events in complex
scenes. Two classes of models for physical scene understanding have recently
been proposed: "Intuitive Physics Engines", or IPEs, which posit that people
make predictions by running approximate probabilistic simulations in causal
mental models similar in nature to video-game physics engines, and memory-based
models, which make judgments based on analogies to stored experiences of
previously encountered scenes and physical outcomes. Versions of the latter
have recently been instantiated in convolutional neural network (CNN)
architectures. Here we report four experiments that, to our knowledge, are the
first rigorous comparisons of simulation-based and CNN-based models, where both
approaches are concretely instantiated in algorithms that can run on raw image
inputs and produce as outputs physical judgments such as whether a stack of
blocks will fall. Both approaches can achieve super-human accuracy levels and
can quantitatively predict human judgments to a similar degree, but only the
simulation-based models generalize to novel situations in ways that people do,
and are qualitatively consistent with systematic perceptual illusions and
judgment asymmetries that people show.

positive:
There is a brief description of the probabilistic causal graph model for
representing, reasoning with, and learning causal structure using Bayesian
networks. It is then argued that this model is closely related to how humans
reason with and learn causal structure. It is shown that studies in psychology
on discounting (reasoning concerning how the presence of one cause of an effect
makes another cause less probable) support the hypothesis that humans reach the
same judgments as algorithms for doing inference in Bayesian networks. Next, it
is shown how studies by Piaget indicate that humans learn causal structure by
observing the same independencies and dependencies as those used by certain
algorithms for learning the structure of a Bayesian network. Based on this
indication, a subjective definition of causality is forwarded. Finally, methods
for further testing the accuracy of these claims are discussed.

positive:
Prevalent theories in cognitive science propose that humans understand and
represent the knowledge of the world through causal relationships. In making
sense of the world, we build causal models in our mind to encode cause-effect
relations of events and use these to explain why new events happen. In this
paper, we use causal models to derive causal explanations of behaviour of
reinforcement learning agents. We present an approach that learns a structural
causal model during reinforcement learning and encodes causal relationships
between variables of interest. This model is then used to generate explanations
of behaviour based on counterfactual analysis of the causal model. We report on
a study with 120 participants who observe agents playing a real-time strategy
game (Starcraft II) and then receive explanations of the agents' behaviour. We
investigated: 1) participants' understanding gained by explanations through
task prediction; 2) explanation satisfaction and 3) trust. Our results show
that causal model explanations perform better on these measures compared to two
other baseline explanation models.

positive:
Probabilistic machine learning models are often insufficient to help with
decisions on interventions because those models find correlations - not causal
relationships. If observational data is only available and experimentation are
infeasible, the correct approach to study the impact of an intervention is to
invoke Pearl's causality framework. Even that framework assumes that the
underlying causal graph is known, which is seldom the case in practice. When
the causal structure is not known, one may use out-of-the-box algorithms to
find causal dependencies from observational data. However, there exists no
method that also accounts for the decision-maker's prior knowledge when
developing the causal structure either. The objective of this paper is to
develop rational approaches for making decisions from observational data in the
presence of causal graph uncertainty and prior knowledge from the
decision-maker. We use ensemble methods like Bayesian Model Averaging (BMA) to
infer set of causal graphs that can represent the data generation process. We
provide decisions by computing the expected value and risk of potential
interventions explicitly. We demonstrate our approach by applying them in
different example contexts.

positive:
Causal reasoning is the main learning and explanation tool used by humans. AI
systems should possess causal reasoning capabilities to be deployed in the real
world with trust and reliability. Introducing the ideas of causality to machine
learning helps in providing better learning and explainable models.
Explainability, causal disentanglement are some important aspects of any
machine learning model. Causal explanations are required to believe in a
model's decision and causal disentanglement learning is important for transfer
learning applications. We exploit the ideas of causality to be used in deep
learning models to achieve better and causally explainable models that are
useful in fairness, disentangled representation, etc.

positive:
State-of-the-art recommender systems have the ability to generate
high-quality recommendations, but usually cannot provide intuitive explanations
to humans due to the usage of black-box prediction models. The lack of
transparency has highlighted the critical importance of improving the
explainability of recommender systems. In this paper, we propose to extract
causal rules from the user interaction history as post-hoc explanations for the
black-box sequential recommendation mechanisms, whilst maintain the predictive
accuracy of the recommendation model. Our approach firstly achieves
counterfactual examples with the aid of a perturbation model, and then extracts
personalized causal relationships for the recommendation model through a causal
rule mining algorithm. Experiments are conducted on several state-of-the-art
sequential recommendation models and real-world datasets to verify the
performance of our model on generating causal explanations. Meanwhile, We
evaluate the discovered causal explanations in terms of quality and fidelity,
which show that compared with conventional association rules, causal rules can
provide personalized and more effective explanations for the behavior of
black-box recommendation models.

positive:
Those designing autonomous systems that interact with humans will invariably
face questions about how humans think and make decisions. Fortunately,
computational cognitive science offers insight into human decision-making using
tools that will be familiar to those with backgrounds in optimization and
control (e.g., probability theory, statistical machine learning, and
reinforcement learning). Here, we review some of this work, focusing on how
cognitive science can provide forward models of human decision-making and
inverse models of how humans think about others' decision-making. We highlight
relevant recent developments, including approaches that synthesize blackbox and
theory-driven modeling, accounts that recast heuristics and biases as forms of
bounded optimality, and models that characterize human theory of mind and
communication in decision-theoretic terms. In doing so, we aim to provide
readers with a glimpse of the range of frameworks, methodologies, and
actionable insights that lie at the intersection of cognitive science and
control research.

positive:
Complex machine learning models are deployed in several critical domains
including healthcare and autonomous vehicles nowadays, albeit as functional
black boxes. Consequently, there has been a recent surge in interpreting
decisions of such complex models in order to explain their actions to humans.
Models that correspond to human interpretation of a task are more desirable in
certain contexts and can help attribute liability, build trust, expose biases
and in turn build better models. It is, therefore, crucial to understand how
and which models conform to human understanding of tasks. In this paper, we
present a large-scale crowdsourcing study that reveals and quantifies the
dissonance between human and machine understanding, through the lens of an
image classification task. In particular, we seek to answer the following
questions: Which (well-performing) complex ML models are closer to humans in
their use of features to make accurate predictions? How does task difficulty
affect the feature selection capability of machines in comparison to humans?
Are humans consistently better at selecting features that make image
recognition more accurate? Our findings have important implications on
human-machine collaboration, considering that a long term goal in the field of
artificial intelligence is to make machines capable of learning and reasoning
like humans.

negative:
Decision-making under uncertainty and causal thinking are fundamental aspects
of intelligent reasoning. Decision-making has been well studied when the
available information is considered at the associative (probabilistic) level.
The classical Theorems of von Neumann-Morgenstern and Savage provide a formal
criterion for rational choice using associative information: maximize expected
utility. There is an ongoing debate around the origin of probabilities involved
in such calculation. In this work, we will show how the probabilities for
decision-making can be grounded in causal models by considering decision
problems in which the available actions and consequences are causally
connected. In this setting, actions are regarded as an intervention over a
causal model. Then, we extend a previous causal decision-making result, which
relies on a known causal model, to the case in which the causal mechanism that
controls some environment is unknown to a rational decision-maker. In this way,
action-outcome probabilities can be grounded in causal models in known and
unknown cases. Finally, as an application, we extend the well-known concept of
Nash Equilibrium to the case in which the players of a strategic game consider
causal information.

negative:
Machine learning has made major advances in categorizing objects in images,
yet the best algorithms miss important aspects of how people learn and think
about categories. People can learn richer concepts from fewer examples,
including causal models that explain how members of a category are formed.
Here, we explore the limits of this human ability to infer causal "programs" --
latent generating processes with nontrivial algorithmic properties -- from one,
two, or three visual examples. People were asked to extrapolate the programs in
several ways, for both classifying and generating new examples. As a theory of
these inductive abilities, we present a Bayesian program learning model that
searches the space of programs for the best explanation of the observations.
Although variable, people's judgments are broadly consistent with the model and
inconsistent with several alternatives, including a pre-trained deep neural
network for object recognition, indicating that people can learn and reason
with rich algorithmic abstractions from sparse input data.

negative:
Discovering and exploiting the causal structure in the environment is a
crucial challenge for intelligent agents. Here we explore whether causal
reasoning can emerge via meta-reinforcement learning. We train a recurrent
network with model-free reinforcement learning to solve a range of problems
that each contain causal structure. We find that the trained agent can perform
causal reasoning in novel situations in order to obtain rewards. The agent can
select informative interventions, draw causal inferences from observational
data, and make counterfactual predictions. Although established formal causal
reasoning algorithms also exist, in this paper we show that such reasoning can
arise from model-free reinforcement learning, and suggest that causal reasoning
in complex settings may benefit from the more end-to-end learning-based
approaches presented here. This work also offers new strategies for structured
exploration in reinforcement learning, by providing agents with the ability to
perform -- and interpret -- experiments.

negative:
The ability to reason about temporal and causal events from videos lies at
the core of human intelligence. Most video reasoning benchmarks, however, focus
on pattern recognition from complex visual and language input, instead of on
causal structure. We study the complementary problem, exploring the temporal
and causal structures behind videos of objects with simple visual appearance.
To this end, we introduce the CoLlision Events for Video REpresentation and
Reasoning (CLEVRER), a diagnostic video dataset for systematic evaluation of
computational models on a wide range of reasoning tasks. Motivated by the
theory of human casual judgment, CLEVRER includes four types of questions:
descriptive (e.g., "what color"), explanatory ("what is responsible for"),
predictive ("what will happen next"), and counterfactual ("what if"). We
evaluate various state-of-the-art models for visual reasoning on our benchmark.
While these models thrive on the perception-based task (descriptive), they
perform poorly on the causal tasks (explanatory, predictive and
counterfactual), suggesting that a principled approach for causal reasoning
should incorporate the capability of both perceiving complex visual and
language inputs, and understanding the underlying dynamics and causal
relations. We also study an oracle model that explicitly combines these
components via symbolic representations.

negative:
The study and understanding of human behaviour is relevant to computer
science, artificial intelligence, neural computation, cognitive science,
philosophy, psychology, and several other areas. Presupposing cognition as
basis of behaviour, among the most prominent tools in the modelling of
behaviour are computational-logic systems, connectionist models of cognition,
and models of uncertainty. Recent studies in cognitive science, artificial
intelligence, and psychology have produced a number of cognitive models of
reasoning, learning, and language that are underpinned by computation. In
addition, efforts in computer science research have led to the development of
cognitive computational systems integrating machine learning and automated
reasoning. Such systems have shown promise in a range of applications,
including computational biology, fault diagnosis, training and assessment in
simulators, and software verification. This joint survey reviews the personal
ideas and views of several researchers on neural-symbolic learning and
reasoning. The article is organised in three parts: Firstly, we frame the scope
and goals of neural-symbolic computation and have a look at the theoretical
foundations. We then proceed to describe the realisations of neural-symbolic
computation, systems, and applications. Finally we present the challenges
facing the area and avenues for further research.

negative:
Recent work in explanation generation for decision making agents has looked
at how unexplained behavior of autonomous systems can be understood in terms of
differences in the model of the system and the human's understanding of the
same, and how the explanation process as a result of this mismatch can be then
seen as a process of reconciliation of these models. Existing algorithms in
such settings, while having been built on contrastive, selective and social
properties of explanations as studied extensively in the psychology literature,
have not, to the best of our knowledge, been evaluated in settings with actual
humans in the loop. As such, the applicability of such explanations to human-AI
and human-robot interactions remains suspect. In this paper, we set out to
evaluate these explanation generation algorithms in a series of studies in a
mock search and rescue scenario with an internal semi-autonomous robot and an
external human commander. We demonstrate to what extent the properties of these
algorithms hold as they are evaluated by humans, and how the dynamics of trust
between the human and the robot evolve during the process of these
interactions.

negative:
The recent success of Bayesian methods in neuroscience and artificial
intelligence gives rise to the hypothesis that the brain is a Bayesian machine.
Since logic and learning are both practices of the human brain, it leads to
another hypothesis that there is a Bayesian interpretation underlying both
logical reasoning and machine learning. In this paper, we introduce a
generative model of logical consequence relations. It formalises the process of
how the truth value of a sentence is probabilistically generated from the
probability distribution over states of the world. We show that the generative
model characterises a classical consequence relation, paraconsistent
consequence relation and nonmonotonic consequence relation. In particular, the
generative model gives a new consequence relation that outperforms them in
reasoning with inconsistent knowledge. We also show that the generative model
gives a new classification algorithm that outperforms several representative
algorithms in predictive accuracy and complexity on the Kaggle Titanic dataset.

negative:
Humans are able to perceive, understand and reason about causal events.
Developing models with similar physical and causal understanding capabilities
is a long-standing goal of artificial intelligence. As a step towards this
direction, we introduce CRAFT, a new video question answering dataset that
requires causal reasoning about physical forces and object interactions. It
contains 58K video and question pairs that are generated from 10K videos from
20 different virtual environments, containing various objects in motion that
interact with each other and the scene. Two question categories in CRAFT
include previously studied descriptive and counterfactual questions.
Additionally, inspired by the Force Dynamics Theory in cognitive linguistics,
we introduce a new causal question category that involves understanding the
causal interactions between objects through notions like cause, enable, and
prevent. Our results show that even though the questions in CRAFT are easy for
humans, the tested baseline models, including existing state-of-the-art
methods, do not yet deal with the challenges posed in our benchmark.

negative:
Causal induction, i.e., identifying unobservable mechanisms that lead to the
observable relations among variables, has played a pivotal role in modern
scientific discovery, especially in scenarios with only sparse and limited
data. Humans, even young toddlers, can induce causal relationships surprisingly
well in various settings despite its notorious difficulty. However, in contrast
to the commonplace trait of human cognition is the lack of a diagnostic
benchmark to measure causal induction for modern Artificial Intelligence (AI)
systems. Therefore, in this work, we introduce the Abstract Causal REasoning
(ACRE) dataset for systematic evaluation of current vision systems in causal
induction. Motivated by the stream of research on causal discovery in Blicket
experiments, we query a visual reasoning system with the following four types
of questions in either an independent scenario or an interventional scenario:
direct, indirect, screening-off, and backward-blocking, intentionally going
beyond the simple strategy of inducing causal relationships by covariation. By
analyzing visual reasoning architectures on this testbed, we notice that pure
neural models tend towards an associative strategy under their chance-level
performance, whereas neuro-symbolic combinations struggle in backward-blocking
reasoning. These deficiencies call for future research in models with a more
comprehensive capability of causal induction.

negative:
Human understanding of narrative is mainly driven by reasoning about causal
relations between events and thus recognizing them is a key capability for
computational models of language understanding. Computational work in this area
has approached this via two different routes: by focusing on acquiring a
knowledge base of common causal relations between events, or by attempting to
understand a particular story or macro-event, along with its storyline. In this
position paper, we focus on knowledge acquisition approach and claim that
newswire is a relatively poor source for learning fine-grained causal relations
between everyday events. We describe experiments using an unsupervised method
to learn causal relations between events in the narrative genres of
first-person narratives and film scene descriptions. We show that our method
learns fine-grained causal relations, judged by humans as likely to be causal
over 80% of the time. We also demonstrate that the learned event pairs do not
exist in publicly available event-pair datasets extracted from newswire.

negative:
We propose to explain the behavior of black-box prediction methods (e.g.,
deep neural networks trained on image pixel data) using causal graphical
models. Specifically, we explore learning the structure of a causal graph where
the nodes represent prediction outcomes along with a set of macro-level
"interpretable" features, while allowing for arbitrary unmeasured confounding
among these variables. The resulting graph may indicate which of the
interpretable features, if any, are possible causes of the prediction outcome
and which may be merely associated with prediction outcomes due to confounding.
The approach is motivated by a counterfactual theory of causal explanation
wherein good explanations point to factors that are "difference-makers" in an
interventionist sense. The resulting analysis may be useful in algorithm
auditing and evaluation, by identifying features which make a causal difference
to the algorithm's output.

negative:
We present a randomized controlled trial for a model-in-the-loop regression
task, with the goal of measuring the extent to which (1) good explanations of
model predictions increase human accuracy, and (2) faulty explanations decrease
human trust in the model. We study explanations based on visual saliency in an
image-based age prediction task for which humans and learned models are
individually capable but not highly proficient and frequently disagree. Our
experimental design separates model quality from explanation quality, and makes
it possible to compare treatments involving a variety of explanations of
varying levels of quality. We find that presenting model predictions improves
human accuracy. However, visual explanations of various kinds fail to
significantly alter human accuracy or trust in the model - regardless of
whether explanations characterize an accurate model, an inaccurate one, or are
generated randomly and independently of the input image. These findings suggest
the need for greater evaluation of explanations in downstream decision making
tasks, better design-based tools for presenting explanations to users, and
better approaches for generating explanations.

negative:
Recent advances in computational cognitive science (i.e., simulation-based
probabilistic programs) have paved the way for significant progress in formal,
implementable models of pragmatics. Rather than describing a pragmatic
reasoning process in prose, these models formalize and implement one, deriving
both qualitative and quantitative predictions of human behavior -- predictions
that consistently prove correct, demonstrating the viability and value of the
framework. The current paper provides a practical introduction to and critical
assessment of the Bayesian Rational Speech Act modeling framework, unpacking
theoretical foundations, exploring technological innovations, and drawing
connections to issues beyond current applications.

negative:
Research in social psychology has shown that people's biased, subjective
judgments about another's personality based solely on their appearance are not
predictive of their actual personality traits. But researchers and companies
often utilize computer vision models to predict similarly subjective
personality attributes such as "employability." We seek to determine whether
state-of-the-art, black box face processing technology can learn human-like
appearance biases. With features extracted with FaceNet, a widely used face
recognition framework, we train a transfer learning model on human subjects'
first impressions of personality traits in other faces as measured by social
psychologists. We find that features extracted with FaceNet can be used to
predict human appearance bias scores for deliberately manipulated faces but not
for randomly generated faces scored by humans. Additionally, in contrast to
work with human biases in social psychology, the model does not find a
significant signal correlating politicians' vote shares with perceived
competence bias. With Local Interpretable Model-Agnostic Explanations (LIME),
we provide several explanations for this discrepancy. Our results suggest that
some signals of appearance bias documented in social psychology are not
embedded by the machine learning techniques we investigate. We shed light on
the ways in which appearance bias could be embedded in face processing
technology and cast further doubt on the practice of predicting subjective
traits based on appearances.

negative:
Despite the surprising power of many modern AI systems that often learn their
own representations, there is significant discontent about their inscrutability
and the attendant problems in their ability to interact with humans. While
alternatives such as neuro-symbolic approaches have been proposed, there is a
lack of consensus on what they are about. There are often two independent
motivations (i) symbols as a lingua franca for human-AI interaction and (ii)
symbols as system-produced abstractions used by the AI system in its internal
reasoning. The jury is still out on whether AI systems will need to use symbols
in their internal reasoning to achieve general intelligence capabilities.
Whatever the answer there is, the need for (human-understandable) symbols in
human-AI interaction seems quite compelling. Symbols, like emotions, may well
not be sine qua non for intelligence per se, but they will be crucial for AI
systems to interact with us humans -- as we can neither turn off our emotions
nor get by without our symbols. In particular, in many human-designed domains,
humans would be interested in providing explicit (symbolic) knowledge and
advice -- and expect machine explanations in kind. This alone requires AI
systems to to maintain a symbolic interface for interaction with humans. In
this blue sky paper, we argue this point of view, and discuss research
directions that need to be pursued to allow for this type of human-AI
interaction.

negative:
When AI systems interact with humans in the loop, they are often called on to
provide explanations for their plans and behavior. Past work on plan
explanations primarily involved the AI system explaining the correctness of its
plan and the rationale for its decision in terms of its own model. Such
soliloquy is wholly inadequate in most realistic scenarios where the humans
have domain and task models that differ significantly from that used by the AI
system. We posit that the explanations are best studied in light of these
differing models. In particular, we show how explanation can be seen as a
"model reconciliation problem" (MRP), where the AI system in effect suggests
changes to the human's model, so as to make its plan be optimal with respect to
that changed human model. We will study the properties of such explanations,
present algorithms for automatically computing them, and evaluate the
performance of the algorithms.

negative:
Uncovering causal relationships in data is a major objective of data
analytics. Causal relationships are normally discovered with designed
experiments, e.g. randomised controlled trials, which, however are expensive or
infeasible to be conducted in many cases. Causal relationships can also be
found using some well designed observational studies, but they require domain
experts' knowledge and the process is normally time consuming. Hence there is a
need for scalable and automated methods for causal relationship exploration in
data. Classification methods are fast and they could be practical substitutes
for finding causal signals in data. However, classification methods are not
designed for causal discovery and a classification method may find false causal
signals and miss the true ones. In this paper, we develop a causal decision
tree where nodes have causal interpretations. Our method follows a well
established causal inference framework and makes use of a classic statistical
test. The method is practical for finding causal signals in large data sets.

negative:
The burgeoning of AI has prompted recommendations that AI techniques should
be "human-centered". However, there is no clear definition of what is meant by
Human Centered Artificial Intelligence, or for short, HCAI. This paper aims to
improve this situation by addressing some foundational aspects of HCAI. To do
so, we introduce the term HCAI agent to refer to any physical or software
computational agent equipped with AI components and that interacts and/or
collaborates with humans. This article identifies five main conceptual
components that participate in an HCAI agent: Observations, Requirements,
Actions, Explanations and Models. We see the notion of HCAI agent, together
with its components and functions, as a way to bridge the technical and
non-technical discussions on human-centered AI. In this paper, we focus our
analysis on scenarios consisting of a single agent operating in dynamic
environments in presence of humans.

negative:
Human decision-making underlies all economic behavior. For the past four
decades, human decision-making under uncertainty has continued to be explained
by theoretical models based on prospect theory, a framework that was awarded
the Nobel Prize in Economic Sciences. However, theoretical models of this kind
have developed slowly, and robust, high-precision predictive models of human
decisions remain a challenge. While machine learning is a natural candidate for
solving these problems, it is currently unclear to what extent it can improve
predictions obtained by current theories. We argue that this is mainly due to
data scarcity, since noisy human behavior requires massive sample sizes to be
accurately captured by off-the-shelf machine learning methods. To solve this
problem, what is needed are machine learning models with appropriate inductive
biases for capturing human behavior, and larger datasets. We offer two
contributions towards this end: first, we construct "cognitive model priors" by
pretraining neural networks with synthetic data generated by cognitive models
(i.e., theoretical models developed by cognitive psychologists). We find that
fine-tuning these networks on small datasets of real human decisions results in
unprecedented state-of-the-art improvements on two benchmark datasets. Second,
we present the first large-scale dataset for human decision-making, containing
over 240,000 human judgments across over 13,000 decision problems. This dataset
reveals the circumstances where cognitive model priors are useful, and provides
a new standard for benchmarking prediction of human decisions under
uncertainty.

negative:
With the rise of machines to human-level performance in complex recognition
tasks, a growing amount of work is directed towards comparing information
processing in humans and machines. These studies are an exciting chance to
learn about one system by studying the other. Here, we propose ideas on how to
design, conduct and interpret experiments such that they adequately support the
investigation of mechanisms when comparing human and machine perception. We
demonstrate and apply these ideas through three case studies. The first case
study shows how human bias can affect how we interpret results, and that
several analytic tools can help to overcome this human reference point. In the
second case study, we highlight the difference between necessary and sufficient
mechanisms in visual reasoning tasks. Thereby, we show that contrary to
previous suggestions, feedback mechanisms might not be necessary for the tasks
in question. The third case study highlights the importance of aligning
experimental conditions. We find that a previously-observed difference in
object recognition does not hold when adapting the experiment to make
conditions more equitable between humans and machines. In presenting a
checklist for comparative studies of visual reasoning in humans and machines,
we hope to highlight how to overcome potential pitfalls in design or inference.

negative:
The problem of replicating the flexibility of human common-sense reasoning
has captured the imagination of computer scientists since the early days of
Alan Turing's foundational work on computation and the philosophy of artificial
intelligence. In the intervening years, the idea of cognition as computation
has emerged as a fundamental tenet of Artificial Intelligence (AI) and
cognitive science. But what kind of computation is cognition?
  We describe a computational formalism centered around a probabilistic Turing
machine called QUERY, which captures the operation of probabilistic
conditioning via conditional simulation. Through several examples and analyses,
we demonstrate how the QUERY abstraction can be used to cast common-sense
reasoning as probabilistic inference in a statistical model of our observations
and the uncertain structure of the world that generated that experience. This
formulation is a recent synthesis of several research programs in AI and
cognitive science, but it also represents a surprising convergence of several
of Turing's pioneering insights in AI, the foundations of computation, and
statistics.

negative:
As AI systems demonstrate increasingly strong predictive performance, their
adoption has grown in numerous domains. However, in high-stakes domains such as
criminal justice and healthcare, full automation is often not desirable due to
safety, ethical, and legal concerns, yet fully manual approaches can be
inaccurate and time consuming. As a result, there is growing interest in the
research community to augment human decision making with AI assistance. Besides
developing AI technologies for this purpose, the emerging field of human-AI
decision making must embrace empirical approaches to form a foundational
understanding of how humans interact and work with AI to make decisions. To
invite and help structure research efforts towards a science of understanding
and improving human-AI decision making, we survey recent literature of
empirical human-subject studies on this topic. We summarize the study design
choices made in over 100 papers in three important aspects: (1) decision tasks,
(2) AI models and AI assistance elements, and (3) evaluation metrics. For each
aspect, we summarize current trends, discuss gaps in current practices of the
field, and make a list of recommendations for future research. Our survey
highlights the need to develop common frameworks to account for the design and
research spaces of human-AI decision making, so that researchers can make
rigorous choices in study design, and the research community can build on each
other's work and produce generalizable scientific knowledge. We also hope this
survey will serve as a bridge for HCI and AI communities to work together to
mutually shape the empirical science and computational technologies for
human-AI decision making.

negative:
Understanding the fundamentals of human reasoning is central to the
development of any system built to closely interact with humans. Cognitive
science pursues the goal of modeling human-like intelligence from a
theory-driven perspective with a strong focus on explainability. Syllogistic
reasoning as one of the core domains of human reasoning research has seen a
surge of computational models being developed over the last years. However,
recent analyses of models' predictive performances revealed a stagnation in
improvement. We believe that most of the problems encountered in cognitive
science are not due to the specific models that have been developed but can be
traced back to the peculiarities of behavioral data instead.
  Therefore, we investigate potential data-related reasons for the problems in
human reasoning research by comparing model performances on human and
artificially generated datasets. In particular, we apply collaborative
filtering recommenders to investigate the adversarial effects of
inconsistencies and noise in data and illustrate the potential for data-driven
methods in a field of research predominantly concerned with gaining high-level
theoretical insight into a domain.
  Our work (i) provides insight into the levels of noise to be expected from
human responses in reasoning data, (ii) uncovers evidence for an upper-bound of
performance that is close to being reached urging for an extension of the
modeling task, and (iii) introduces the tools and presents initial results to
pioneer a new paradigm for investigating and modeling reasoning focusing on
predicting responses for individual human reasoners.

negative:
Human collaborators can effectively communicate with their partners to finish
a common task by inferring each other's mental states (e.g., goals, beliefs,
and desires). Such mind-aware communication minimizes the discrepancy among
collaborators' mental states, and is crucial to the success in human ad-hoc
teaming. We believe that robots collaborating with human users should
demonstrate similar pedagogic behavior. Thus, in this paper, we propose a novel
explainable AI (XAI) framework for achieving human-like communication in
human-robot collaborations, where the robot builds a hierarchical mind model of
the human user and generates explanations of its own mind as a form of
communications based on its online Bayesian inference of the user's mental
state. To evaluate our framework, we conduct a user study on a real-time
human-robot cooking task. Experimental results show that the generated
explanations of our approach significantly improves the collaboration
performance and user perception of the robot. Code and video demos are
available on our project website: https://xfgao.github.io/xCookingWeb/.

negative:
When human cognition is modeled in Philosophy and Cognitive Science, there is
a pervasive idea that humans employ mental representations in order to navigate
the world and make predictions about outcomes of future actions. By
understanding how these representational structures work, we not only
understand more about human cognition but also gain a better understanding for
how humans rationalise and explain decisions. This has an influencing effect on
explainable AI, where the goal is to provide explanations of computer
decision-making for a human audience. We show that the Contextual Importance
and Utility method for XAI share an overlap with the current new wave of
action-oriented predictive representational structures, in ways that makes CIU
a reliable tool for creating explanations that humans can relate to and trust.

negative:
We discuss the changes in an attitude to decision making at the fire ground.
The changes are driven by the recent technological shift. The emerging new
approaches in sensing and data processing (under common umbrella of
Cyber-Physical Systems) allow for leveling off the gap, between humans and
machines, in perception of the fire ground. Furthermore, results from
descriptive decision theory question the rationality of human choices. This
creates the need for searching and testing new approaches for decision making
during emergency. We propose the framework that addresses this need. The
primary feature of the framework are possibilities for incorporation of
normative and prescriptive approaches to decision making. The framework also
allows for comparison of the performance of decisions, between human and
machine.

negative:
Bayesian networks can be used to extract explanations about the observed
state of a subset of variables. In this paper, we explicate the desiderata of
an explanation and confront them with the concept of explanation proposed by
existing methods. The necessity of taking into account causal approaches when a
causal graph is available is discussed. We then introduce causal explanation
trees, based on the construction of explanation trees using the measure of
causal information ow (Ay and Polani, 2006). This approach is compared to
several other methods on known networks.

negative:
Higher-level cognition depends on the ability to learn models of the world.
We can characterize this at the computational level as a structure-learning
problem with the goal of best identifying the prevailing causal relationships
among a set of relata. However, the computational cost of performing exact
Bayesian inference over causal models grows rapidly as the number of relata
increases. This implies that the cognitive processes underlying causal learning
must be substantially approximate. A powerful class of approximations that
focuses on the sequential absorption of successive inputs is captured by the
Neurath's ship metaphor in philosophy of science, where theory change is cast
as a stochastic and gradual process shaped as much by people's limited
willingness to abandon their current theory when considering alternatives as by
the ground truth they hope to approach. Inspired by this metaphor and by
algorithms for approximating Bayesian inference in machine learning, we propose
an algorithmic-level model of causal structure learning under which learners
represent only a single global hypothesis that they update locally as they
gather evidence. We propose a related scheme for understanding how, under these
limitations, learners choose informative interventions that manipulate the
causal system to help elucidate its workings. We find support for our approach
in the analysis of four experiments.

negative:
A large body of work in behavioral fields attempts to develop models that
describe the way people, as opposed to rational agents, make decisions. A
recent Choice Prediction Competition (2015) challenged researchers to suggest a
model that captures 14 classic choice biases and can predict human decisions
under risk and ambiguity. The competition focused on simple decision problems,
in which human subjects were asked to repeatedly choose between two gamble
options.
  In this paper we present our approach for predicting human decision behavior:
we suggest to use machine learning algorithms with features that are based on
well-established behavioral theories. The basic idea is that these
psychological features are essential for the representation of the data and are
important for the success of the learning process. We implement a vanilla model
in which we train SVM models using behavioral features that rely on the
psychological properties underlying the competition baseline model. We show
that this basic model captures the 14 choice biases and outperforms all the
other learning-based models in the competition. The preliminary results suggest
that such hybrid models can significantly improve the prediction of human
decision making, and are a promising direction for future research.

negative:
Causal inference is central to many areas of artificial intelligence,
including complex reasoning, planning, knowledge-base construction, robotics,
explanation, and fairness. An active community of researchers develops and
enhances algorithms that learn causal models from data, and this work has
produced a series of impressive technical advances. However, evaluation
techniques for causal modeling algorithms have remained somewhat primitive,
limiting what we can learn from experimental studies of algorithm performance,
constraining the types of algorithms and model representations that researchers
consider, and creating a gap between theory and practice. We argue for more
frequent use of evaluation techniques that examine interventional measures
rather than structural or observational measures, and that evaluate those
measures on empirical data rather than synthetic data. We survey the current
practice in evaluation and show that the techniques we recommend are rarely
used in practice. We show that such techniques are feasible and that data sets
are available to conduct such evaluations. We also show that these techniques
produce substantially different results than using structural measures and
synthetic data.

negative:
Recent progress in artificial intelligence (AI) has renewed interest in
building systems that learn and think like people. Many advances have come from
using deep neural networks trained end-to-end in tasks such as object
recognition, video games, and board games, achieving performance that equals or
even beats humans in some respects. Despite their biological inspiration and
performance achievements, these systems differ from human intelligence in
crucial ways. We review progress in cognitive science suggesting that truly
human-like learning and thinking machines will have to reach beyond current
engineering trends in both what they learn, and how they learn it.
Specifically, we argue that these machines should (a) build causal models of
the world that support explanation and understanding, rather than merely
solving pattern recognition problems; (b) ground learning in intuitive theories
of physics and psychology, to support and enrich the knowledge that is learned;
and (c) harness compositionality and learning-to-learn to rapidly acquire and
generalize knowledge to new tasks and situations. We suggest concrete
challenges and promising routes towards these goals that can combine the
strengths of recent neural network advances with more structured cognitive
models.

negative:
The ability to learn and act in novel situations is still a prerogative of
animate intelligence, as current machine learning methods mostly fail when
moving beyond the standard i.i.d. setting. What is the reason for this
discrepancy? Most machine learning tasks are anti-causal, i.e., we infer causes
(labels) from effects (observations). Typically, in supervised learning we
build systems that try to directly invert causal mechanisms. Instead, in this
paper we argue that strong generalization capabilities crucially hinge on
searching and validating meaningful hypotheses, requiring access to a causal
model. In such a framework, we want to find a cause that leads to the observed
effect. Anti-causal models are used to drive this search, but a causal model is
required for validation. We investigate the fundamental differences between
causal and anti-causal tasks, discuss implications for topics ranging from
adversarial attacks to disentangling factors of variation, and provide
extensive evidence from the literature to substantiate our view. We advocate
for incorporating causal models in supervised learning to shift the paradigm
from inference only, to search and validation.

negative:
Human beings learn causal models and constantly use them to transfer
knowledge between similar environments. We use this intuition to design a
transfer-learning framework using object-oriented representations to learn the
causal relationships between objects. A learned causal dynamics model can be
used to transfer between variants of an environment with exchangeable
perceptual features among objects but with the same underlying causal dynamics.
We adapt continuous optimization for structure learning techniques to
explicitly learn the cause and effects of the actions in an interactive
environment and transfer to the target domain by categorization of the objects
based on causal knowledge. We demonstrate the advantages of our approach in a
gridworld setting by combining causal model-based approach with model-free
approach in reinforcement learning.

negative:
We consider decision problems under uncertainty where the options available
to a decision maker and the resulting outcome are related through a causal
mechanism which is unknown to the decision maker. We ask how a decision maker
can learn about this causal mechanism through sequential decision making as
well as using current causal knowledge inside each round in order to make
better choices had she not considered causal knowledge and propose a decision
making procedure in which an agent holds \textit{beliefs} about her environment
which are used to make a choice and are updated using the observed outcome. As
proof of concept, we present an implementation of this causal decision making
model and apply it in a simple scenario. We show that the model achieves a
performance similar to the classic Q-learning while it also acquires a causal
model of the environment.

negative:
Causal reasoning has been an indispensable capability for humans and other
intelligent animals to interact with the physical world. In this work, we
propose to endow an artificial agent with the capability of causal reasoning
for completing goal-directed tasks. We develop learning-based approaches to
inducing causal knowledge in the form of directed acyclic graphs, which can be
used to contextualize a learned goal-conditional policy to perform tasks in
novel environments with latent causal structures. We leverage attention
mechanisms in our causal induction model and goal-conditional policy, enabling
us to incrementally generate the causal graph from the agent's visual
observations and to selectively use the induced graph for determining actions.
Our experiments show that our method effectively generalizes towards completing
new tasks in novel environments with previously unseen causal structures.

negative:
Linear structural causal models (SCMs) -- in which each observed variable is
generated by a subset of the other observed variables as well as a subset of
the exogenous sources -- are pervasive in causal inference and casual
discovery. However, for the task of causal discovery, existing work almost
exclusively focus on the submodel where each observed variable is associated
with a distinct source with non-zero variance. This results in the restriction
that no observed variable can deterministically depend on other observed
variables or latent confounders. In this paper, we extend the results on
structure learning by focusing on a subclass of linear SCMs which do not have
this property, i.e., models in which observed variables can be causally
affected by any subset of the sources, and are allowed to be a deterministic
function of other observed variables or latent confounders. This allows for a
more realistic modeling of influence or information propagation in systems. We
focus on the task of causal discovery form observational data generated from a
member of this subclass. We derive a set of necessary and sufficient conditions
for unique identifiability of the causal structure. To the best of our
knowledge, this is the first work that gives identifiability results for causal
discovery under both latent confounding and deterministic relationships.
Further, we propose an algorithm for recovering the underlying causal structure
when the aforementioned conditions are satisfied. We validate our theoretical
results both on synthetic and real datasets.

negative:
Understanding predictions made by deep neural networks is notoriously
difficult, but also crucial to their dissemination. As all machine learning
based methods, they are as good as their training data, and can also capture
unwanted biases. While there are tools that can help understand whether such
biases exist, they do not distinguish between correlation and causation, and
might be ill-suited for text-based models and for reasoning about high level
language concepts. A key problem of estimating the causal effect of a concept
of interest on a given model is that this estimation requires the generation of
counterfactual examples, which is challenging with existing generation
technology. To bridge that gap, we propose CausaLM, a framework for producing
causal model explanations using counterfactual language representation models.
Our approach is based on fine-tuning of deep contextualized embedding models
with auxiliary adversarial tasks derived from the causal graph of the problem.
Concretely, we show that by carefully choosing auxiliary adversarial
pre-training tasks, language representation models such as BERT can effectively
learn a counterfactual representation for a given concept of interest, and be
used to estimate its true causal effect on model performance. A byproduct of
our method is a language representation model that is unaffected by the tested
concept, which can be useful in mitigating unwanted bias ingrained in the data.

negative:
Intelligence can be defined as a predominantly human ability to accomplish
tasks that are generally hard for computers and animals. Artificial
Intelligence [AI] is a field attempting to accomplish such tasks with
computers. AI is becoming increasingly widespread, as are claims of its
relationship with Biological Intelligence. Often these claims are made to imply
higher chances of a given technology succeeding, working on the assumption that
AI systems which mimic the mechanisms of Biological Intelligence should be more
successful.
  In this article I will discuss the similarities and differences between AI
and the extent of our knowledge about the mechanisms of intelligence in
biology, especially within humans. I will also explore the validity of the
assumption that biomimicry in AI systems aids their advancement, and I will
argue that existing similarity to biological systems in the way Artificial
Neural Networks [ANNs] tackle tasks is due to design decisions, rather than
inherent similarity of underlying mechanisms. This article is aimed at people
who understand the basics of AI (especially ANNs), and would like to be better
able to evaluate the often wild claims about the value of biomimicry in AI.

negative:
Human social behavior is structured by relationships. We form teams, groups,
tribes, and alliances at all scales of human life. These structures guide
multi-agent cooperation and competition, but when we observe others these
underlying relationships are typically unobservable and hence must be inferred.
Humans make these inferences intuitively and flexibly, often making rapid
generalizations about the latent relationships that underlie behavior from just
sparse and noisy observations. Rapid and accurate inferences are important for
determining who to cooperate with, who to compete with, and how to cooperate in
order to compete. Towards the goal of building machine-learning algorithms with
human-like social intelligence, we develop a generative model of multi-agent
action understanding based on a novel representation for these latent
relationships called Composable Team Hierarchies (CTH). This representation is
grounded in the formalism of stochastic games and multi-agent reinforcement
learning. We use CTH as a target for Bayesian inference yielding a new
algorithm for understanding behavior in groups that can both infer hidden
relationships as well as predict future actions for multiple agents interacting
together. Our algorithm rapidly recovers an underlying causal model of how
agents relate in spatial stochastic games from just a few observations. The
patterns of inference made by this algorithm closely correspond with human
judgments and the algorithm makes the same rapid generalizations that people
do.

negative:
In this paper, we aim to develop a unified view of causal and non-causal
feature selection methods. The unified view will fill in the gap in the
research of the relation between the two types of methods. Based on the
Bayesian network framework and information theory, we first show that causal
and non-causal feature selection methods share the same objective. That is to
find the Markov blanket of a class attribute, the theoretically optimal feature
set for classification. We then examine the assumptions made by causal and
non-causal feature selection methods when searching for the optimal feature
set, and unify the assumptions by mapping them to the restrictions on the
structure of the Bayesian network model of the studied problem. We further
analyze in detail how the structural assumptions lead to the different levels
of approximations employed by the methods in their search, which then result in
the approximations in the feature sets found by the methods with respect to the
optimal feature set. With the unified view, we are able to interpret the output
of non-causal methods from a causal perspective and derive the error bounds of
both types of methods. Finally, we present practical understanding of the
relation between causal and non-causal methods using extensive experiments with
synthetic data and various types of real-word data.

negative:
We propose CX-ToM, short for counterfactual explanations with theory-of mind,
a new explainable AI (XAI) framework for explaining decisions made by a deep
convolutional neural network (CNN). In contrast to the current methods in XAI
that generate explanations as a single shot response, we pose explanation as an
iterative communication process, i.e. dialog, between the machine and human
user. More concretely, our CX-ToM framework generates sequence of explanations
in a dialog by mediating the differences between the minds of machine and human
user. To do this, we use Theory of Mind (ToM) which helps us in explicitly
modeling human's intention, machine's mind as inferred by the human as well as
human's mind as inferred by the machine. Moreover, most state-of-the-art XAI
frameworks provide attention (or heat map) based explanations. In our work, we
show that these attention based explanations are not sufficient for increasing
human trust in the underlying CNN model. In CX-ToM, we instead use
counterfactual explanations called fault-lines which we define as follows:
given an input image I for which a CNN classification model M predicts class
c_pred, a fault-line identifies the minimal semantic-level features (e.g.,
stripes on zebra, pointed ears of dog), referred to as explainable concepts,
that need to be added to or deleted from I in order to alter the classification
category of I by M to another specified class c_alt. We argue that, due to the
iterative, conceptual and counterfactual nature of CX-ToM explanations, our
framework is practical and more natural for both expert and non-expert users to
understand the internal workings of complex deep learning models. Extensive
quantitative and qualitative experiments verify our hypotheses, demonstrating
that our CX-ToM significantly outperforms the state-of-the-art explainable AI
models.

negative:
Humans make complex inferences on faces, ranging from objective properties
(gender, ethnicity, expression, age, identity, etc) to subjective judgments
(facial attractiveness, trustworthiness, sociability, friendliness, etc). While
the objective aspects of face perception have been extensively studied,
relatively fewer computational models have been developed for the social
impressions of faces. Bridging this gap, we develop a method to predict human
impressions of faces in 40 subjective social dimensions, using deep
representations from state-of-the-art neural networks. We find that model
performance grows as the human consensus on a face trait increases, and that
model predictions outperform human groups in correlation with human averages.
This illustrates the learnability of subjective social perception of faces,
especially when there is high human consensus. Our system can be used to decide
which photographs from a personal collection will make the best impression. The
results are significant for the field of social robotics, demonstrating that
robots can learn the subjective judgments defining the underlying fabric of
human interaction.

negative:
Neural networks have long been used to model human intelligence, capturing
elements of behavior and cognition, and their neural basis. Recent advancements
in deep learning have enabled neural network models to reach and even surpass
human levels of intelligence in many respects, yet unlike humans, their ability
to learn new tasks quickly remains a challenge. People can reason not only in
familiar domains, but can also rapidly learn to reason through novel problems
and situations, raising the question of how well modern neural network models
capture human intelligence and in which ways they diverge. In this work, we
explore this gap by investigating human adults' ability to learn an abstract
reasoning task based on Sudoku from a brief instructional tutorial with
explanatory feedback for incorrect responses using a narrow range of training
examples. We find that participants who master the task do so within a small
number of trials and generalize well to puzzles outside of the training range.
We also find that most of those who master the task can describe a valid
solution strategy, and such participants perform better on transfer puzzles
than those whose strategy descriptions are vague or incomplete. Interestingly,
fewer than half of our human participants were successful in acquiring a valid
solution strategy, and this ability is associated with high school mathematics
education. We consider the challenges these findings pose for building
computational models that capture all aspects of our findings and point toward
a possible role for learning to engage in explanation-based reasoning to
support rapid learning and generalization.

negative:
As environments involving both robots and humans become increasingly common,
so does the need to account for people during planning. To plan effectively,
robots must be able to respond to and sometimes influence what humans do. This
requires a human model which predicts future human actions. A simple model may
assume the human will continue what they did previously; a more complex one
might predict that the human will act optimally, disregarding the robot;
whereas an even more complex one might capture the robot's ability to influence
the human. These models make different trade-offs between computational time
and performance of the resulting robot plan. Using only one model of the human
either wastes computational resources or is unable to handle critical
situations. In this work, we give the robot access to a suite of human models
and enable it to assess the performance-computation trade-off online. By
estimating how an alternate model could improve human prediction and how that
may translate to performance gain, the robot can dynamically switch human
models whenever the additional computation is justified. Our experiments in a
driving simulator showcase how the robot can achieve performance comparable to
always using the best human model, but with greatly reduced computation.

negative:
This document is written with the intention to describe in detail a method
and means by which a computer program can reason about the world and in so
doing, increase its analogue to a living system. As the literature is rife and
it is apparent we, as scientists and engineers, have not found the solution,
this document will attempt the solution by grounding its intellectual arguments
within tenets of human cognition in Western philosophy. The result will be a
characteristic description of a method to describe an artificial system
analogous to that performed for a human. The approach was the substance of my
Master's thesis, explored more deeply during the course of my postdoc research.
It focuses primarily on context awareness and choice set within a boundary of
available epistemology, which serves to describe it. Expanded upon, such a
description strives to discover agreement with Kant's critique of reason to
understand how it could be applied to define the architecture of its design.
The intention has never been to mimic human or biological systems, rather, to
understand the profoundly fundamental rules, when leveraged correctly, results
in an artificial consciousness as noumenon while in keeping with the perception
of it as phenomenon.

negative:
This paper proposes a research direction to advance AI which draws
inspiration from cognitive theories of human decision making. The premise is
that if we gain insights about the causes of some human capabilities that are
still lacking in AI (for instance, adaptability, generalizability, common
sense, and causal reasoning), we may obtain similar capabilities in an AI
system by embedding these causal components. We hope that the high-level
description of our vision included in this paper, as well as the several
research questions that we propose to consider, can stimulate the AI research
community to define, try and evaluate new methodologies, frameworks, and
evaluation metrics, in the spirit of achieving a better understanding of both
human and machine intelligence.

negative:
Causal Learner is a toolbox for learning causal structure and Markov blanket
(MB) from data. It integrates functions for generating simulated Bayesian
network data, a set of state-of-the-art global causal structure learning
algorithms, a set of state-of-the-art local causal structure learning
algorithms, a set of state-of-the-art MB learning algorithms, and functions for
evaluating algorithms. The data generation part of Causal Learner is written in
R, and the rest of Causal Learner is written in MATLAB. Causal Learner aims to
provide researchers and practitioners with an open-source platform for causal
learning from data and for the development and evaluation of new causal
learning algorithms. The Causal Learner project is available at
http://bigdata.ahu.edu.cn/causal-learner.

negative:
We consider the incorporation of causal knowledge about the presence or
absence of (possibly indirect) causal relations into a causal model. Such
causal relations correspond to directed paths in a causal model. This type of
knowledge naturally arises from experimental data, among others. Specifically,
we consider the formalisms of Causal Bayesian Networks and Maximal Ancestral
Graphs and their Markov equivalence classes: Partially Directed Acyclic Graphs
and Partially Oriented Ancestral Graphs. We introduce sound and complete
procedures which are able to incorporate causal prior knowledge in such models.
In simulated experiments, we show that often considering even a few causal
facts leads to a significant number of new inferences. In a case study, we also
show how to use real experimental data to infer causal knowledge and
incorporate it into a real biological causal network. The code is available at
mensxmachina.org.

negative:
Deep neural networks have become increasingly successful at solving classic
perception problems such as object recognition, semantic segmentation, and
scene understanding, often reaching or surpassing human-level accuracy. This
success is due in part to the ability of DNNs to learn useful representations
of high-dimensional inputs, a problem that humans must also solve. We examine
the relationship between the representations learned by these networks and
human psychological representations recovered from similarity judgments. We
find that deep features learned in service of object classification account for
a significant amount of the variance in human similarity judgments for a set of
animal images. However, these features do not capture some qualitative
distinctions that are a key part of human representations. To remedy this, we
develop a method for adapting deep features to align with human similarity
judgments, resulting in image representations that can potentially be used to
extend the scope of psychological experiments.

negative:
In this paper we propose a causal analog to the purely observational Dynamic
Bayesian Networks, which we call Dynamic Causal Networks. We provide a sound
and complete algorithm for identification of Dynamic Causal Net- works, namely,
for computing the effect of an intervention or experiment, based on passive
observations only, whenever possible. We note the existence of two types of
confounder variables that affect in substantially different ways the iden-
tification procedures, a distinction with no analog in either Dynamic Bayesian
Networks or standard causal graphs. We further propose a procedure for the
transportability of causal effects in Dynamic Causal Network settings, where
the re- sult of causal experiments in a source domain may be used for the
identification of causal effects in a target domain.

negative:
Causal discovery, the task of automatically constructing a causal model from
data, is of major significance across the sciences. Evaluating the performance
of causal discovery algorithms should ideally involve comparing the inferred
models to ground-truth models available for benchmark datasets, which in turn
requires a notion of distance between causal models. While such distances have
been proposed previously, they are limited by focusing on graphical properties
of the causal models being compared. Here, we overcome this limitation by
defining distances derived from the causal distributions induced by the models,
rather than exclusively from their graphical structure. Pearl and Mackenzie
(2018) have arranged the properties of causal models in a hierarchy called the
"ladder of causation" spanning three rungs: observational, interventional, and
counterfactual. Following this organization, we introduce a hierarchy of three
distances, one for each rung of the ladder. Our definitions are intuitively
appealing as well as efficient to compute approximately. We put our causal
distances to use by benchmarking standard causal discovery systems on both
synthetic and real-world datasets for which ground-truth causal models are
available. Finally, we highlight the usefulness of our causal distances by
briefly discussing further applications beyond the evaluation of causal
discovery techniques.

negative:
Probabilistic graphical models such as Bayesian Networks are one of the most
powerful structures known by the Computer Science community for deriving
probabilistic inferences. However, modern cognitive psychology has revealed
that human decisions could not follow the rules of classical probability
theory, because humans cannot process large amounts of data in order to make
judgements. Consequently, the inferences performed are based on limited data
coupled with several heuristics, leading to violations of the law of total
probability. This means that probabilistic graphical models based on classical
probability theory are too limited to fully simulate and explain various
aspects of human decision making.
  Quantum probability theory was developed in order to accommodate the
paradoxical findings that the classical theory could not explain. Recent
findings in cognitive psychology revealed that quantum probability can fully
describe human decisions in an elegant framework. Their findings suggest that,
before taking a decision, human thoughts are seen as superposed waves that can
interfere with each other, influencing the final decision.
  In this work, we propose a new Bayesian Network based on the psychological
findings of cognitive scientists. We made experiments with two very well known
Bayesian Networks from the literature. The results obtained revealed that the
quantum like Bayesian Network can affect drastically the probabilistic
inferences, specially when the levels of uncertainty of the network are very
high (no pieces of evidence observed). When the levels of uncertainty are very
low, then the proposed quantum like network collapses to its classical
counterpart.

negative:
While feedback loops are known to play important roles in many complex
systems, their existence is ignored in a large part of the causal discovery
literature, as systems are typically assumed to be acyclic from the outset.
When applying causal discovery algorithms designed for the acyclic setting on
data generated by a system that involves feedback, one would not expect to
obtain correct results. In this work, we show that---surprisingly---the output
of the Fast Causal Inference (FCI) algorithm is correct if it is applied to
observational data generated by a system that involves feedback. More
specifically, we prove that for observational data generated by a simple and
$\sigma$-faithful Structural Causal Model (SCM), FCI is sound and complete, and
can be used to consistently estimate (i) the presence and absence of causal
relations, (ii) the presence and absence of direct causal relations, (iii) the
absence of confounders, and (iv) the absence of specific cycles in the causal
graph of the SCM. We extend these results to constraint-based causal discovery
algorithms that exploit certain forms of background knowledge, including the
causally sufficient setting (e.g., the PC algorithm) and the Joint Causal
Inference setting (e.g., the FCI-JCI algorithm).

negative:
How can humans remain in control of artificial intelligence (AI)-based
systems designed to perform tasks autonomously? Such systems are increasingly
ubiquitous, creating benefits - but also undesirable situations where moral
responsibility for their actions cannot be properly attributed to any
particular person or group. The concept of meaningful human control has been
proposed to address responsibility gaps and mitigate them by establishing
conditions that enable a proper attribution of responsibility for humans;
however, clear requirements for researchers, designers, and engineers are yet
inexistent, making the development of AI-based systems that remain under
meaningful human control challenging. In this paper, we address the gap between
philosophical theory and engineering practice by identifying, through an
iterative process of abductive thinking, four actionable properties for
AI-based systems under meaningful human control, which we discuss making use of
two applications scenarios: automated vehicles and AI-based hiring. First, a
system in which humans and AI algorithms interact should have an explicitly
defined domain of morally loaded situations within which the system ought to
operate. Second, humans and AI agents within the system should have appropriate
and mutually compatible representations. Third, responsibility attributed to a
human should be commensurate with that human's ability and authority to control
the system. Fourth, there should be explicit links between the actions of the
AI agents and actions of humans who are aware of their moral responsibility. We
argue that these four properties will support practically-minded professionals
to take concrete steps toward designing and engineering for AI systems that
facilitate meaningful human control.

negative:
Counterfactual inference is a powerful tool, capable of solving challenging
problems in high-profile sectors. To perform counterfactual inference, one
requires knowledge of the underlying causal mechanisms. However, causal
mechanisms cannot be uniquely determined from observations and interventions
alone. This raises the question of how to choose the causal mechanisms so that
resulting counterfactual inference is trustworthy in a given domain. This
question has been addressed in causal models with binary variables, but the
case of categorical variables remains unanswered. We address this challenge by
introducing for causal models with categorical variables the notion of
counterfactual ordering, a principle that posits desirable properties causal
mechanisms should posses, and prove that it is equivalent to specific
functional constraints on the causal mechanisms. To learn causal mechanisms
satisfying these constraints, and perform counterfactual inference with them,
we introduce deep twin networks. These are deep neural networks that, when
trained, are capable of twin network counterfactual inference -- an alternative
to the abduction, action, & prediction method. We empirically test our approach
on diverse real-world and semi-synthetic data from medicine, epidemiology, and
finance, reporting accurate estimation of counterfactual probabilities while
demonstrating the issues that arise with counterfactual reasoning when
counterfactual ordering is not enforced.

negative:
Humans learn complex latent structures from their environments (e.g., natural
language, mathematics, music, social hierarchies). In cognitive science and
cognitive neuroscience, models that infer higher-order structures from sensory
or first-order representations have been proposed to account for the complexity
and flexibility of human behavior. But how do the structures that these models
invoke arise in neural systems in the first place? To answer this question, we
explain how a system can learn latent representational structures (i.e.,
predicates) from experience with wholly unstructured data. During the process
of predicate learning, an artificial neural network exploits the naturally
occurring dynamic properties of distributed computing across neuronal
assemblies in order to learn predicates, but also to combine them
compositionally, two computational aspects which appear to be necessary for
human behavior as per formal theories in multiple domains. We describe how
predicates can be combined generatively using neural oscillations to achieve
human-like extrapolation and compositionality in an artificial neural network.
The ability to learn predicates from experience, to represent structures
compositionally, and to extrapolate to unseen data offers an inroads to
understanding and modeling the most complex human behaviors.

negative:
This paper describes a Bayesian method for combining an arbitrary mixture of
observational and experimental data in order to learn causal Bayesian networks.
Observational data are passively observed. Experimental data, such as that
produced by randomized controlled trials, result from the experimenter
manipulating one or more variables (typically randomly) and observing the
states of other variables. The paper presents a Bayesian method for learning
the causal structure and parameters of the underlying causal process that is
generating the data, given that (1) the data contains a mixture of
observational and experimental case records, and (2) the causal process is
modeled as a causal Bayesian network. This learning method was applied using as
input various mixtures of experimental and observational data that were
generated from the ALARM causal Bayesian network. In these experiments, the
absolute and relative quantities of experimental and observational data were
varied systematically. For each of these training datasets, the learning method
was applied to predict the causal structure and to estimate the causal
parameters that exist among randomly selected pairs of nodes in ALARM that are
not confounded. The paper reports how these structure predictions and parameter
estimates compare with the true causal structures and parameters as given by
the ALARM network.

negative:
Experiential AI is proposed as a new research agenda in which artists and
scientists come together to dispel the mystery of algorithms and make their
mechanisms vividly apparent. It addresses the challenge of finding novel ways
of opening up the field of artificial intelligence to greater transparency and
collaboration between human and machine. The hypothesis is that art can mediate
between computer code and human comprehension to overcome the limitations of
explanations in and for AI systems. Artists can make the boundaries of systems
visible and offer novel ways to make the reasoning of AI transparent and
decipherable. Beyond this, artistic practice can explore new configurations of
humans and algorithms, mapping the terrain of inter-agencies between people and
machines. This helps to viscerally understand the complex causal chains in
environments with AI components, including questions about what data to collect
or who to collect it about, how the algorithms are chosen, commissioned and
configured or how humans are conditioned by their participation in algorithmic
processes.

negative:
In attempts to produce ML models less reliant on spurious patterns in NLP
datasets, researchers have recently proposed curating counterfactually
augmented data (CAD) via a human-in-the-loop process in which given some
documents and their (initial) labels, humans must revise the text to make a
counterfactual label applicable. Importantly, edits that are not necessary to
flip the applicable label are prohibited. Models trained on the augmented data
appear, empirically, to rely less on semantically irrelevant words and to
generalize better out of domain. While this work draws loosely on causal
thinking, the underlying causal model (even at an abstract level) and the
principles underlying the observed out-of-domain improvements remain unclear.
In this paper, we introduce a toy analog based on linear Gaussian models,
observing interesting relationships between causal models, measurement noise,
out-of-domain generalization, and reliance on spurious signals. Our analysis
provides some insights that help to explain the efficacy of CAD. Moreover, we
develop the hypothesis that while adding noise to causal features should
degrade both in-domain and out-of-domain performance, adding noise to
non-causal features should lead to relative improvements in out-of-domain
performance. This idea inspires a speculative test for determining whether a
feature attribution technique has identified the causal spans. If adding noise
(e.g., by random word flips) to the highlighted spans degrades both in-domain
and out-of-domain performance on a battery of challenge datasets, but adding
noise to the complement gives improvements out-of-domain, it suggests we have
identified causal spans. We present a large-scale empirical study comparing
spans edited to create CAD to those selected by attention and saliency maps.
Across numerous domains and models, we find that the hypothesized phenomenon is
pronounced for CAD.

negative:
This paper presents a sound and completecalculus for causal relevance, based
onPearl's functional models semantics.The calculus consists of axioms and
rulesof inference for reasoning about causalrelevance relationships.We extend
the set of known axioms for causalrelevance with three new axioms, andintroduce
two new rules of inference forreasoning about specific subclasses
ofmodels.These subclasses give a more refinedcharacterization of causal models
than the one given in Halpern's axiomatizationof counterfactual
reasoning.Finally, we show how the calculus for causalrelevance can be used in
the task ofidentifying causal structure from non-observational data.

negative:
Probability trees are one of the simplest models of causal generative
processes. They possess clean semantics and -- unlike causal Bayesian networks
-- they can represent context-specific causal dependencies, which are necessary
for e.g. causal induction. Yet, they have received little attention from the AI
and ML community. Here we present concrete algorithms for causal reasoning in
discrete probability trees that cover the entire causal hierarchy (association,
intervention, and counterfactuals), and operate on arbitrary propositional and
causal events. Our work expands the domain of causal reasoning to a very
general class of discrete stochastic processes.

negative:
In this work we propose a multi-valued extension of logic programs under the
stable models semantics where each true atom in a model is associated with a
set of justifications. These justifications are expressed in terms of causal
graphs formed by rule labels and edges that represent their application
ordering. For positive programs, we show that the causal justifications
obtained for a given atom have a direct correspon- dence to (relevant)
syntactic proofs of that atom using the program rules involved in the graphs.
The most interesting contribution is that this causal information is obtained
in a purely semantic way, by algebraic op- erations (product, sum and
application) on a lattice of causal values whose ordering relation expresses
when a justification is stronger than another. Finally, for programs with
negation, we define the concept of causal stable model by introducing an
analogous transformation to Gelfond and Lifschitz's program reduct. As a
result, default negation behaves as "absence of proof" and no justification is
derived from negative liter

negative:
A neuroscience method to understanding the brain is to find and study the
preferred stimuli that highly activate an individual cell or groups of cells.
Recent advances in machine learning enable a family of methods to synthesize
preferred stimuli that cause a neuron in an artificial or biological brain to
fire strongly. Those methods are known as Activation Maximization (AM) or
Feature Visualization via Optimization. In this chapter, we (1) review existing
AM techniques in the literature; (2) discuss a probabilistic interpretation for
AM; and (3) review the applications of AM in debugging and explaining networks.

negative:
Automation of tasks can have critical consequences when humans lose agency
over decision processes. Deep learning models are particularly susceptible
since current black-box approaches lack explainable reasoning. We argue that
both the visual interface and model structure of deep learning systems need to
take into account interaction design. We propose a framework of collaborative
semantic inference (CSI) for the co-design of interactions and models to enable
visual collaboration between humans and algorithms. The approach exposes the
intermediate reasoning process of models which allows semantic interactions
with the visual metaphors of a problem, which means that a user can both
understand and control parts of the model reasoning process. We demonstrate the
feasibility of CSI with a co-designed case study of a document summarization
system.

negative:
A Random Graph is a random object which take its values in the space of
graphs. We take advantage of the expressibility of graphs in order to model the
uncertainty about the existence of causal relationships within a given set of
variables. We adopt a Bayesian point of view in order to capture a causal
structure via interaction and learning with a causal environment. We test our
method over two different scenarios, and the experiments mainly confirm that
our technique can learn a causal structure. Furthermore, the experiments and
results presented for the first test scenario demonstrate the usefulness of our
method to learn a causal structure as well as the optimal action. On the other
hand the second experiment, shows that our proposal manages to learn the
underlying causal structure of several tasks with different sizes and different
causal structures.

negative:
This paper discusses causal independence models and a generalization of these
models called causal interaction models. Causal interaction models are models
that have independent mechanisms where a mechanism can have several causes. In
addition to introducing several particular types of causal interaction models,
we show how we can apply the Bayesian approach to learning causal interaction
models obtaining approximate posterior distributions for the models and obtain
MAP and ML estimates for the parameters. We illustrate the approach with a
simulation study of learning model posteriors.

negative:
Standard automatic metrics, e.g. BLEU, are not reliable for document-level MT
evaluation. They can neither distinguish document-level improvements in
translation quality from sentence-level ones, nor identify the discourse
phenomena that cause context-agnostic translations. This paper introduces a
novel automatic metric BlonDe to widen the scope of automatic MT evaluation
from sentence to document level. BlonDe takes discourse coherence into
consideration by categorizing discourse-related spans and calculating the
similarity-based F1 measure of categorized spans. We conduct extensive
comparisons on a newly constructed dataset BWB. The experimental results show
that BlonDe possesses better selectivity and interpretability at the
document-level, and is more sensitive to document-level nuances. In a
large-scale human study, BlonDe also achieves significantly higher Pearson's r
correlation with human judgments compared to previous metrics.

negative:
This paper proposes a causal inference relation and causal programming as
general frameworks for causal inference with structural causal models. A tuple,
$\langle M, I, Q, F \rangle$, is an instance of the relation if a formula, $F$,
computes a causal query, $Q$, as a function of known population probabilities,
$I$, in every model entailed by a set of model assumptions, $M$. Many problems
in causal inference can be viewed as the problem of enumerating instances of
the relation that satisfy given criteria. This unifies a number of previously
studied problems, including causal effect identification, causal discovery and
recovery from selection bias. In addition, the relation supports formalizing
new problems in causal inference with structural causal models, such as the
problem of research design. Causal programming is proposed as a further
generalization of causal inference as the problem of finding optimal instances
of the relation, with respect to a cost function.

negative:
The act of explaining across two parties is a feedback loop, where one
provides information on what needs to be explained and the other provides an
explanation relevant to this information. We apply a reinforcement learning
framework which emulates this format by providing explanations based on the
explainee's current mental model. We conduct novel online human experiments
where explanations generated by various explanation methods are selected and
presented to participants, using policies which observe participants' mental
models, in order to optimize an interpretability proxy. Our results suggest
that mental model-based policies (anchored in our proposed state
representation) may increase interpretability over multiple sequential
explanations, when compared to a random selection baseline. This work provides
insight into how to select explanations which increase relevant information for
users, and into conducting human-grounded experimentation to understand
interpretability.

negative:
We present novel automatic metrics for machine translation evaluation that
use discourse structure and convolution kernels to compare the discourse tree
of an automatic translation with that of the human reference. We experiment
with five transformations and augmentations of a base discourse tree
representation based on the rhetorical structure theory, and we combine the
kernel scores for each of them into a single score. Finally, we add other
metrics from the ASIYA MT evaluation toolkit, and we tune the weights of the
combination on actual human judgments. Experiments on the WMT12 and WMT13
metrics shared task datasets show correlation with human judgments that
outperforms what the best systems that participated in these years achieved,
both at the segment and at the system level.

negative:
The ability of humans to quickly identify general concepts from a handful of
images has proven difficult to emulate with robots. Recently, a computer
architecture was developed that allows robots to mimic some aspects of this
human ability by modeling concepts as cognitive programs using an instruction
set of primitive cognitive functions. This allowed a robot to emulate human
imagination by simulating candidate programs in a world model before
generalizing to the physical world. However, this model used a naive search
algorithm that required 30 minutes to discover a single concept, and became
intractable for programs with more than 20 instructions. To circumvent this
bottleneck, we present an algorithm that emulates the human cognitive
heuristics of object factorization and sub-goaling, allowing human-level
inference speed, improving accuracy, and making the output more explainable.

negative:
This paper introduces Whittemore, a language for causal programming. Causal
programming is based on the theory of structural causal models and consists of
two primary operations: identification, which finds formulas that compute
causal queries, and estimation, which applies formulas to transform probability
distributions to other probability distribution. Causal programming provides
abstractions to declare models, queries, and distributions with syntax similar
to standard mathematical notation, and conducts rigorous causal inference,
without requiring detailed knowledge of the underlying algorithms. Examples of
causal inference with real data are provided, along with discussion of the
implementation and possibilities for future extension.

negative:
Representation learning is a key element of state-of-the-art deep learning
approaches. It enables to transform raw data into structured vector space
embeddings. Such embeddings are able to capture the distributional semantics of
their context, e.g. by word windows on natural language sentences, graph walks
on knowledge graphs or convolutions on images. So far, this context is manually
defined, resulting in heuristics which are solely optimized for computational
performance on certain tasks like link-prediction. However, such heuristic
models of context are fundamentally different to how humans capture
information. For instance, when reading a multi-modal webpage (i) humans do not
perceive all parts of a document equally: Some words and parts of images are
skipped, others are revisited several times which makes the perception trace
highly non-sequential; (ii) humans construct meaning from a document's content
by shifting their attention between text and image, among other things, guided
by layout and design elements. In this paper we empirically investigate the
difference between human perception and context heuristics of basic embedding
models. We conduct eye tracking experiments to capture the underlying
characteristics of human perception of media documents containing a mixture of
text and images. Based on that, we devise a prototypical computational
perception-trace model, called CMPM. We evaluate empirically how CMPM can
improve a basic skip-gram embedding approach. Our results suggest, that even
with a basic human-inspired computational perception model, there is a huge
potential for improving embeddings since such a model does inherently capture
multiple modalities, as well as layout and design elements.

negative:
We fine-tune GPT-3 to answer long-form questions using a text-based
web-browsing environment, which allows the model to search and navigate the
web. By setting up the task so that it can be performed by humans, we are able
to train models on the task using imitation learning, and then optimize answer
quality with human feedback. To make human evaluation of factual accuracy
easier, models must collect references while browsing in support of their
answers. We train and evaluate our models on ELI5, a dataset of questions asked
by Reddit users. Our best model is obtained by fine-tuning GPT-3 using behavior
cloning, and then performing rejection sampling against a reward model trained
to predict human preferences. This model's answers are preferred by humans 56%
of the time to those of our human demonstrators, and 69% of the time to the
highest-voted answer from Reddit.

negative:
A structural causal model is made of endogenous (manifest) and exogenous
(latent) variables. We show that endogenous observations induce linear
constraints on the probabilities of the exogenous variables. This allows to
exactly map a causal model into a credal network. Causal inferences, such as
interventions and counterfactuals, can consequently be obtained by standard
algorithms for the updating of credal nets. These natively return sharp values
in the identifiable case, while intervals corresponding to the exact bounds are
produced for unidentifiable queries. A characterization of the causal models
that allow the map above to be compactly derived is given, along with a
discussion about the scalability for general models. This contribution should
be regarded as a systematic approach to represent structural causal models by
credal networks and hence to systematically compute causal inferences. A number
of demonstrative examples is presented to clarify our methodology. Extensive
experiments show that approximate algorithms for credal networks can
immediately be used to do causal inference in real-size problems.

negative:
We define a Causal Decision Problem as a Decision Problem where the available
actions, the family of uncertain events and the set of outcomes are related
through the variables of a Causal Graphical Model $\mathcal{G}$. A solution
criteria based on Pearl's Do-Calculus and the Expected Utility criteria for
rational preferences is proposed. The implementation of this criteria leads to
an on-line decision making procedure that has been shown to have similar
performance to classic Reinforcement Learning algorithms while allowing for a
causal model of an environment to be learned. Thus, we aim to provide the
theoretical guarantees of the usefulness and optimality of a decision making
procedure based on causal information.

negative:
We model the behavioral biases of human decision-making in securing
interdependent systems and show that such behavioral decision-making leads to a
suboptimal pattern of resource allocation compared to non-behavioral (rational)
decision-making. We provide empirical evidence for the existence of such
behavioral bias model through a controlled subject study with 145 participants.
We then propose three learning techniques for enhancing decision-making in
multi-round setups. We illustrate the benefits of our decision-making model
through multiple interdependent real-world systems and quantify the level of
gain compared to the case in which the defenders are behavioral. We also show
the benefit of our learning techniques against different attack models. We
identify the effects of different system parameters on the degree of
suboptimality of security outcomes due to behavioral decision-making.

negative:
Interpolation is an important property of classical and many non-classical
logics that has been shown to have interesting applications in computer science
and AI. Here we study the Interpolation Property for the the non-monotonic
system of equilibrium logic, establishing weaker or stronger forms of
interpolation depending on the precise interpretation of the inference
relation. These results also yield a form of interpolation for ground logic
programs under the answer sets semantics. For disjunctive logic programs we
also study the property of uniform interpolation that is closely related to the
concept of variable forgetting. The first-order version of equilibrium logic
has analogous Interpolation properties whenever the collection of equilibrium
models is (first-order) definable. Since this is the case for so-called safe
programs and theories, it applies to the usual situations that arise in
practical answer set programming.

negative:
The encoder-decoder framework achieves state-of-the-art results in keyphrase
generation (KG) tasks by predicting both present keyphrases that appear in the
source document and absent keyphrases that do not. However, relying solely on
the source document can result in generating uncontrollable and inaccurate
absent keyphrases. To address these problems, we propose a novel graph-based
method that can capture explicit knowledge from related references. Our model
first retrieves some document-keyphrases pairs similar to the source document
from a pre-defined index as references. Then a heterogeneous graph is
constructed to capture relationships of different granularities between the
source document and its references. To guide the decoding process, a
hierarchical attention and copy mechanism is introduced, which directly copies
appropriate words from both the source document and its references based on
their relevance and significance. The experimental results on multiple KG
benchmarks show that the proposed model achieves significant improvements
against other baseline models, especially with regard to the absent keyphrase
prediction.

negative:
The humanity has been facing a plethora of challenges associated with
infectious diseases, which kill more than 6 million people a year. Although
continuous efforts have been applied to relieve the potential damages from such
misfortunate events, it is unquestionable that there are many persisting
challenges yet to overcome. One related issue we particularly address here is
the assessment and prediction of such epidemics. In this field of study,
traditional and ad-hoc models frequently fail to provide proper predictive
situation awareness (PSAW), characterized by understanding the current
situations and predicting the future situations. Comprehensive PSAW for
infectious disease can support decision making and help to hinder disease
spread. In this paper, we develop a computing system platform focusing on
collective intelligence causal modeling, in order to support PSAW in the domain
of infectious disease. Analyses of global epidemics require integration of
multiple different data and models, which can be originated from multiple
independent researchers. These models should be integrated to accurately assess
and predict the infectious disease in terms of holistic view. The system shall
provide three main functions: (1) collaborative causal modeling, (2) causal
model integration, and (3) causal model reasoning. These functions are
supported by subject-matter expert and artificial intelligence (AI), with
uncertainty treatment. Subject-matter experts, as collective intelligence,
develop causal models and integrate them as one joint causal model. The
integrated causal model shall be used to reason about: (1) the past, regarding
how the causal factors have occurred; (2) the present, regarding how the spread
is going now; and (3) the future, regarding how it will proceed. Finally, we
introduce one use case of predictive situation awareness for the Ebola virus
disease.

negative:
The causal structure of any system can be analyzed at a multitude of spatial
and temporal scales. It has long been thought that while higher scale (macro)
descriptions of causal structure may be useful to observers, they are at best a
compressed description and at worse leave out critical information. However,
recent research applying information theory to causal analysis has shown that
the causal structure of some systems can actually come into focus (be more
informative) at a macroscale (Hoel et al. 2013). That is, a macro model of a
system (a map) can be more informative than a fully detailed model of the
system (the territory). This has been called causal emergence. While causal
emergence may at first glance seem counterintuitive, this paper grounds the
phenomenon in a classic concept from information theory: Shannon's discovery of
the channel capacity. I argue that systems have a particular causal capacity,
and that different causal models of those systems take advantage of that
capacity to various degrees. For some systems, only macroscale causal models
use the full causal capacity. Such macroscale causal models can either be
coarse-grains, or may leave variables and states out of the model (exogenous)
in various ways, which can improve the model's efficacy and its informativeness
via the same mathematical principles of how error-correcting codes take
advantage of an information channel's capacity. As model choice increase, the
causal capacity of a system approaches the channel capacity. Ultimately, this
provides a general framework for understanding how the causal structure of some
systems cannot be fully captured by even the most detailed microscopic model.

negative:
For robots to be accommodated in human spaces and in humans daily activities,
robots should be able to understand messages from the human conversation
partner. In the same light, humans must also understand the messages that are
being communicated by robots, including the non-verbal ones. We conducted a
web-based video study wherein participants gave interpretations on the iconic
gestures and emblems that were produced by an anthropomorphic robot. Out of the
15 gestures presented, we found 6 robotic gestures that can be accurately
recognized by the human observer. These were nodding, clapping, hugging,
expressing anger, walking, and flying. We reviewed these gestures for their
meaning from literatures in human and animal behavior. We conclude by
discussing the possible implications of these gestures for the design of social
robots that are aimed to have engaging interactions with humans.

negative:
This paper assumes the hypothesis that human learning is perception based,
and consequently, the learning process and perceptions should not be
represented and investigated independently or modeled in different simulation
spaces. In order to keep the analogy between the artificial and human learning,
the former is assumed here as being based on the artificial perception. Hence,
instead of choosing to apply or develop a Computational Theory of (human)
Perceptions, we choose to mirror the human perceptions in a numeric
(computational) space as artificial perceptions and to analyze the
interdependence between artificial learning and artificial perception in the
same numeric space, using one of the simplest tools of Artificial Intelligence
and Soft Computing, namely the perceptrons. As practical applications, we
choose to work around two examples: Optical Character Recognition and Iris
Recognition. In both cases a simple Turing test shows that artificial
perceptions of the difference between two characters and between two irides are
fuzzy, whereas the corresponding human perceptions are, in fact, crisp.

negative:
We extend the definition of the marginal causal effect to the continuous
treatment setting and develop a novel characterization of causal bias in the
framework of structural causal models. We prove that our derived bias
expression is zero if, and only if, the causal effect is identifiable via
covariate adjustment. We show that under some restrictions on the structural
equations, the causal bias can be estimated efficiently and allows for causal
regularization of predictive probabilistic models. We demonstrate the
effectiveness of our method for causal bias quantification in various settings
where (not) controlling for certain covariates would introduce causal bias.

negative:
In recent years, G\"odel's ontological proof and variations of it were
formalized and analyzed with automated tools in various ways. We supplement
these analyses with a modeling in an automated environment based on first-order
logic extended by predicate quantification. Formula macros are used to
structure complex formulas and tasks. The analysis is presented as a generated
type-set document where informal explanations are interspersed with
pretty-printed formulas and outputs of reasoners for first-order theorem
proving and second-order quantifier elimination. Previously unnoticed or
obscured aspects and details of G\"odel's proof become apparent. Practical
application possibilities of second-order quantifier elimination are shown and
the encountered elimination tasks may serve as benchmarks.

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
Untile recently crowdsourcing has been primarily conceived as an online
activity to harness resources for problem solving. However the emergence of
opportunistic networking (ON) has opened up crowdsourcing to the spatial
domain. In this paper we bring the ON model for potential crowdsourcing in the
smart city environment. We introduce cognitive features to the ON that allow
users' mobile devices to become aware of the surrounding physical environment.
Specifically, we exploit cognitive psychology studies on dynamic memory
structures and cognitive heuristics, i.e. mental models that describe how the
human brain handle decision-making amongst complex and real-time stimuli.
Combined with ON, these cognitive features allow devices to act as proxies in
the cyber-world of their users and exchange knowledge to deliver awareness of
places in an urban environment. This is done through tags associated with
locations. They represent features that are perceived by humans about a place.
We consider the extent to which this knowledge becomes available to
participants, using interactions with locations and other nodes. This is
assessed taking into account a wide range of cognitive parameters. Outcomes are
important because this functionality could support a new type of recommendation
system that is independent of the traditional forms of networking.

negative:
We used a token-wise and document-wise sentiment analysis using both
unsupervised and supervised models applied to both news and user reviews
dataset. And our token-wise sentiment analysis found a statistically
significant difference in sentiment between the two groups (both of which were
very large N), our document-wise supervised sentiment analysis found no
significant difference in sentiment.

negative:
This paper describes an application of reinforcement learning to the mention
detection task. We define a novel action-based formulation for the mention
detection task, in which a model can flexibly revise past labeling decisions by
grouping together tokens and assigning partial mention labels. We devise a
method to create mention-level episodes and we train a model by rewarding
correctly labeled complete mentions, irrespective of the inner structure
created. The model yields results which are on par with a competitive
supervised counterpart while being more flexible in terms of achieving targeted
behavior through reward modeling and generating internal mention structure,
especially on longer mentions.

negative:
Neural networks with deep architectures have demonstrated significant
performance improvements in computer vision, speech recognition, and natural
language processing. The challenges in information retrieval (IR), however, are
different from these other application areas. A common form of IR involves
ranking of documents--or short passages--in response to keyword-based queries.
Effective IR systems must deal with query-document vocabulary mismatch problem,
by modeling relationships between different query and document terms and how
they indicate relevance. Models should also consider lexical matches when the
query contains rare terms--such as a person's name or a product model
number--not seen during training, and to avoid retrieving semantically related
but irrelevant results. In many real-life IR tasks, the retrieval involves
extremely large collections--such as the document index of a commercial Web
search engine--containing billions of documents. Efficient IR methods should
take advantage of specialized IR data structures, such as inverted index, to
efficiently retrieve from large collections. Given an information need, the IR
system also mediates how much exposure an information artifact receives by
deciding whether it should be displayed, and where it should be positioned,
among other results. Exposure-aware IR systems may optimize for additional
objectives, besides relevance, such as parity of exposure for retrieved items
and content publishers. In this thesis, we present novel neural architectures
and methods motivated by the specific needs and challenges of IR tasks.

negative:
Named entity linking is to map an ambiguous mention in documents to an entity
in a knowledge base. The named entity linking is challenging, given the fact
that there are multiple candidate entities for a mention in a document. It is
difficult to link a mention when it appears multiple times in a document, since
there are conflicts by the contexts around the appearances of the mention. In
addition, it is difficult since the given training dataset is small due to the
reason that it is done manually to link a mention to its mapping entity. In the
literature, there are many reported studies among which the recent embedding
methods learn vectors of entities from the training dataset at document level.
To address these issues, we focus on how to link entity for mentions at a
sentence level, which reduces the noises introduced by different appearances of
the same mention in a document at the expense of insufficient information to be
used. We propose a new unified embedding method by maximizing the relationships
learned from knowledge graphs. We confirm the effectiveness of our method in
our experimental studies.

negative:
This paper introduces application of Reflexive Game Theory to the matter of
multistage decision making processes. The idea behind is that each decision
making session has certain parameters like "when the session is taking place",
"who are the group members to make decision", "how group members influence on
each other", etc. This study illustrates the consecutive or sequential decision
making process, which consist of two stages. During the stage 1 decisions about
the parameters of the ultimate decision making are made. Then stage 2 is
implementation of Ultimate decision making itself. Since during stage 1 there
can be multiple decision sessions. In such a case it takes more than two
sessions to make ultimate (final) decision. Therefore the overall process of
ultimate decision making becomes multistage decision making process consisting
of consecutive decision making sessions.

negative:
Document date is essential for many important tasks, such as document
retrieval, summarization, event detection, etc. While existing approaches for
these tasks assume accurate knowledge of the document date, this is not always
available, especially for arbitrary documents from the Web. Document Dating is
a challenging problem which requires inference over the temporal structure of
the document. Prior document dating systems have largely relied on handcrafted
features while ignoring such document internal structures. In this paper, we
propose NeuralDater, a Graph Convolutional Network (GCN) based document dating
approach which jointly exploits syntactic and temporal graph structures of
document in a principled way. To the best of our knowledge, this is the first
application of deep learning for the problem of document dating. Through
extensive experiments on real-world datasets, we find that NeuralDater
significantly outperforms state-of-the-art baseline by 19% absolute (45%
relative) accuracy points.