query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : I am seeking a generative modeling approach capable of creating new levels and potentially game settings/environments for a video game with multiple existing levels of difficulty. Specifically, I am interested in exploring how Generative Adversarial Networks (GANs) and other generative methods could generate entirely new levels by emulating the style of previous ones. It is crucial that the newly generated levels are not merely derivative and that my generative model can optimize specific properties, such as the intensity or graphic nature of the game. Given that these properties are non-differentiable, I need a method to either render them differentiable or employ a reinforcement learning-centric approach to optimize these rewards. After generating a variety of levels, I require a method to select some of the best ones. One potential solution could be to evaluate the generated levels using an automatic metric, such as the performance of an AI agent playing the level. Alternatively, I am considering designing a derivative-free stochastic optimization algorithm to guide the search across the space of all synthetically generated levels, steering towards those that meet specific objectives.

positive:
Generative adversarial networks (GANs) are quickly becoming a ubiquitous
approach to procedurally generating video game levels. While GAN generated
levels are stylistically similar to human-authored examples, human designers
often want to explore the generative design space of GANs to extract
interesting levels. However, human designers find latent vectors opaque and
would rather explore along dimensions the designer specifies, such as number of
enemies or obstacles. We propose using state-of-the-art quality diversity
algorithms designed to optimize continuous spaces, i.e. MAP-Elites with a
directional variation operator and Covariance Matrix Adaptation MAP-Elites, to
efficiently explore the latent space of a GAN to extract levels that vary
across a set of specified gameplay measures. In the benchmark domain of Super
Mario Bros, we demonstrate how designers may specify gameplay measures to our
system and extract high-quality (playable) levels with a diverse range of level
mechanics, while still maintaining stylistic similarity to human authored
examples. An online user study shows how the different mechanics of the
automatically generated levels affect subjective ratings of their perceived
difficulty and appearance.

positive:
Several works have demonstrated the use of variational autoencoders (VAEs)
for generating levels in the style of existing games and blending levels across
different games. Further, quality-diversity (QD) algorithms have also become
popular for generating varied game content by using evolution to explore a
search space while focusing on both variety and quality. To reap the benefits
of both these approaches, we present a level generation and game blending
approach that combines the use of VAEs and QD algorithms. Specifically, we
train VAEs on game levels and run the MAP-Elites QD algorithm using the learned
latent space of the VAE as the search space. The latent space captures the
properties of the games whose levels we want to generate and blend, while
MAP-Elites searches this latent space to find a diverse set of levels
optimizing a given objective such as playability. We test our method using
models for 5 different platformer games as well as a blended domain spanning 3
of these games. We refer to using MAP-Elites for blending as Blend-Elites. Our
results show that MAP-Elites in conjunction with VAEs enables the generation of
a diverse set of playable levels not just for each individual game but also for
the blended domain while illuminating game-specific regions of the blended
latent space.

positive:
Recent advancements in procedural content generation via machine learning
enable the generation of video-game levels that are aesthetically similar to
human-authored examples. However, the generated levels are often unplayable
without additional editing. We propose a generate-then-repair framework for
automatic generation of playable levels adhering to specific styles. The
framework constructs levels using a generative adversarial network (GAN)
trained with human-authored examples and repairs them using a mixed-integer
linear program (MIP) with playability constraints. A key component of the
framework is computing minimum cost edits between the GAN generated level and
the solution of the MIP solver, which we cast as a minimum cost network flow
problem. Results show that the proposed framework generates a diverse range of
playable levels, that capture the spatial relationships between objects
exhibited in the human-authored levels.

positive:
Methods for dynamic difficulty adjustment allow games to be tailored to
particular players to maximize their engagement. However, current methods often
only modify a limited set of game features such as the difficulty of the
opponents, or the availability of resources. Other approaches, such as
experience-driven Procedural Content Generation (PCG), can generate complete
levels with desired properties such as levels that are neither too hard nor too
easy, but require many iterations. This paper presents a method that can
generate and search for complete levels with a specific target difficulty in
only a few trials. This advance is enabled by through an Intelligent
Trial-and-Error algorithm, originally developed to allow robots to adapt
quickly. Our algorithm first creates a large variety of different levels that
vary across predefined dimensions such as leniency or map coverage. The
performance of an AI playing agent on these maps gives a proxy for how
difficult the level would be for another AI agent (e.g. one that employs Monte
Carlo Tree Search instead of Greedy Tree Search); using this information, a
Bayesian Optimization procedure is deployed, updating the difficulty of the
prior map to reflect the ability of the agent. The approach can reliably find
levels with a specific target difficulty for a variety of planning agents in
only a few trials, while maintaining an understanding of their skill landscape.

positive:
Deep reinforcement learning (RL) has shown impressive results in a variety of
domains, learning directly from high-dimensional sensory streams. However, when
neural networks are trained in a fixed environment, such as a single level in a
video game, they will usually overfit and fail to generalize to new levels.
When RL models overfit, even slight modifications to the environment can result
in poor agent performance. This paper explores how procedurally generated
levels during training can increase generality. We show that for some games
procedural level generation enables generalization to new levels within the
same distribution. Additionally, it is possible to achieve better performance
with less data by manipulating the difficulty of the levels in response to the
performance of the agent. The generality of the learned behaviors is also
evaluated on a set of human-designed levels. The results suggest that the
ability to generalize to human-designed levels highly depends on the design of
the level generators. We apply dimensionality reduction and clustering
techniques to visualize the generators' distributions of levels and analyze to
what degree they can produce levels similar to those designed by a human.

positive:
Generative Adversarial Networks (GANs) have demonstrated their ability to
learn patterns in data and produce new exemplars similar to, but different
from, their training set in several domains, including video games. However,
GANs have a fixed output size, so creating levels of arbitrary size for a
dungeon crawling game is difficult. GANs also have trouble encoding semantic
requirements that make levels interesting and playable. This paper combines a
GAN approach to generating individual rooms with a graph grammar approach to
combining rooms into a dungeon. The GAN captures design principles of
individual rooms, but the graph grammar organizes rooms into a global layout
with a sequence of obstacles determined by a designer. Room data from The
Legend of Zelda is used to train the GAN. This approach is validated by a user
study, showing that GAN dungeons are as enjoyable to play as a level from the
original game, and levels generated with a graph grammar alone. However, GAN
dungeons have rooms considered more complex, and plain graph grammar's dungeons
are considered least complex and challenging. Only the GAN approach creates an
extensive supply of both layouts and rooms, where rooms span across the
spectrum of those seen in the training set to new creations merging design
principles from multiple rooms.

positive:
Generative Adversarial Networks (GANs) are proving to be a powerful indirect
genotype-to-phenotype mapping for evolutionary search, but they have
limitations. In particular, GAN output does not scale to arbitrary dimensions,
and there is no obvious way of combining multiple GAN outputs into a cohesive
whole, which would be useful in many areas, such as the generation of video
game levels. Game levels often consist of several segments, sometimes repeated
directly or with variation, organized into an engaging pattern. Such patterns
can be produced with Compositional Pattern Producing Networks (CPPNs).
Specifically, a CPPN can define latent vector GAN inputs as a function of
geometry, which provides a way to organize level segments output by a GAN into
a complete level. This new CPPN2GAN approach is validated in both Super Mario
Bros. and The Legend of Zelda. Specifically, divergent search via MAP-Elites
demonstrates that CPPN2GAN can better cover the space of possible levels. The
layouts of the resulting levels are also more cohesive and aesthetically
consistent.

positive:
The procedural generation of levels and content in video games is a
challenging AI problem. Often such generation relies on an intelligent way of
evaluating the content being generated so that constraints are satisfied and/or
objectives maximized. In this work, we address the problem of creating levels
that are not only playable but also revolve around specific mechanics in the
game. We use constrained evolutionary algorithms and quality-diversity
algorithms to generate small sections of Super Mario Bros levels called scenes,
using three different simulation approaches: Limited Agents, Punishing Model,
and Mechanics Dimensions. All three approaches are able to create scenes that
give opportunity for a player to encounter or use targeted mechanics with
different properties. We conclude by discussing the advantages and
disadvantages of each approach and compare them to each other.

positive:
Previous work explored blending levels from existing games to create levels
for a new game that mixes properties of the original games. In this paper, we
use Variational Autoencoders (VAEs) for improving upon such techniques. VAEs
are artificial neural networks that learn and use latent representations of
datasets to generate novel outputs. We train a VAE on level data from Super
Mario Bros. and Kid Icarus, enabling it to capture the latent space spanning
both games. We then use this space to generate level segments that combine
properties of levels from both games. Moreover, by applying evolutionary search
in the latent space, we evolve level segments satisfying specific constraints.
We argue that these affordances make the VAE-based approach especially suitable
for co-creative level design and compare its performance with similar
generative models like the GAN and the VAE-GAN.

positive:
Generative Adversarial Networks (GANs) have shown im-pressive results for
image generation. However, GANs facechallenges in generating contents with
certain types of con-straints, such as game levels. Specifically, it is
difficult togenerate levels that have aesthetic appeal and are playable atthe
same time. Additionally, because training data usually islimited, it is
challenging to generate unique levels with cur-rent GANs. In this paper, we
propose a new GAN architec-ture namedConditional Embedding Self-Attention
Genera-tive Adversarial Network(CESAGAN) and a new bootstrap-ping training
procedure. The CESAGAN is a modification ofthe self-attention GAN that
incorporates an embedding fea-ture vector input to condition the training of
the discriminatorand generator. This allows the network to model
non-localdependency between game objects, and to count objects. Ad-ditionally,
to reduce the number of levels necessary to trainthe GAN, we propose a
bootstrapping mechanism in whichplayable generated levels are added to the
training set. Theresults demonstrate that the new approach does not only
gen-erate a larger number of levels that are playable but also gen-erates fewer
duplicate levels compared to a standard GAN.

positive:
It has recently been shown that reinforcement learning can be used to train
generators capable of producing high-quality game levels, with quality defined
in terms of some user-specified heuristic. To ensure that these generators'
output is sufficiently diverse (that is, not amounting to the reproduction of a
single optimal level configuration), the generation process is constrained such
that the initial seed results in some variance in the generator's output.
However, this results in a loss of control over the generated content for the
human user. We propose to train generators capable of producing controllably
diverse output, by making them "goal-aware." To this end, we add conditional
inputs representing how close a generator is to some heuristic, and also modify
the reward mechanism to incorporate that value. Testing on multiple domains, we
show that the resulting level generators are capable of exploring the space of
possible levels in a targeted, controllable manner, producing levels of
comparable quality as their goal-unaware counterparts, that are diverse along
designer-specified dimensions.

positive:
We investigate how reinforcement learning can be used to train
level-designing agents. This represents a new approach to procedural content
generation in games, where level design is framed as a game, and the content
generator itself is learned. By seeing the design problem as a sequential task,
we can use reinforcement learning to learn how to take the next action so that
the expected final level quality is maximized. This approach can be used when
few or no examples exist to train from, and the trained generator is very fast.
We investigate three different ways of transforming two-dimensional level
design problems into Markov decision processes and apply these to three game
environments.

negative:
This paper introduces a surrogate model of gameplay that learns the mapping
between different game facets, and applies it to a generative system which
designs new content in one of these facets. Focusing on the shooter game genre,
the paper explores how deep learning can help build a model which combines the
game level structure and the game's character class parameters as input and the
gameplay outcomes as output. The model is trained on a large corpus of game
data from simulations with artificial agents in random sets of levels and class
parameters. The model is then used to generate classes for specific levels and
for a desired game outcome, such as balanced matches of short duration.
Findings in this paper show that the system can be expressive and can generate
classes for both computer generated and human authored levels.

negative:
This paper describes a new evolutionary algorithm that is especially well
suited to AI-Assisted Game Design. The approach adopted in this paper is to use
observations of AI agents playing the game to estimate the game's quality. Some
of best agents for this purpose are General Video Game AI agents, since they
can be deployed directly on a new game without game-specific tuning; these
agents tend to be based on stochastic algorithms which give robust but noisy
results and tend to be expensive to run. This motivates the main contribution
of the paper: the development of the novel N-Tuple Bandit Evolutionary
Algorithm, where a model is used to estimate the fitness of unsampled points
and a bandit approach is used to balance exploration and exploitation of the
search space. Initial results on optimising a Space Battle game variant suggest
that the algorithm offers far more robust results than the Random Mutation Hill
Climber and a Biased Mutation variant, which are themselves known to offer
competitive performance across a range of problems. Subjective observations are
also given by human players on the nature of the evolved games, which indicate
a preference towards games generated by the N-Tuple algorithm.

negative:
In this work, we present TOAD-GAN (Token-based One-shot Arbitrary Dimension
Generative Adversarial Network), a novel Procedural Content Generation (PCG)
algorithm that generates token-based video game levels. TOAD-GAN follows the
SinGAN architecture and can be trained using only one example. We demonstrate
its application for Super Mario Bros. levels and are able to generate new
levels of similar style in arbitrary sizes. We achieve state-of-the-art results
in modeling the patterns of the training level and provide a comparison with
different baselines under several metrics. Additionally, we present an
extension of the method that allows the user to control the generation process
of certain token structures to ensure a coherent global level layout. We
provide this tool to the community to spur further research by publishing our
source code.

negative:
Open-endedness, primarily studied in the context of artificial life, is the
ability of systems to generate potentially unbounded ontologies of increasing
novelty and complexity. Engineering generative systems displaying at least some
degree of this ability is a goal with clear applications to procedural content
generation in games. The Paired Open-Ended Trailblazer (POET) algorithm,
heretofore explored only in a biped walking domain, is a coevolutionary system
that simultaneously generates environments and agents that can solve them. This
paper introduces a POET-Inspired Neuroevolutionary System for KreativitY
(PINSKY) in games, which co-generates levels for multiple video games and
agents that play them. This system leverages the General Video Game Artificial
Intelligence (GVGAI) framework to enable co-generation of levels and agents for
the 2D Atari-style games Zelda and Solar Fox. Results demonstrate the ability
of PINSKY to generate curricula of game levels, opening up a promising new
avenue for research at the intersection of procedural content generation and
artificial life. At the same time, results in these challenging game domains
highlight the limitations of the current algorithm and opportunities for
improvement.

negative:
We introduce a procedural content generation (PCG) framework at the
intersections of experience-driven PCG and PCG via reinforcement learning,
named ED(PCG)RL, EDRL in short. EDRL is able to teach RL designers to generate
endless playable levels in an online manner while respecting particular
experiences for the player as designed in the form of reward functions. The
framework is tested initially in the Super Mario Bros game. In particular, the
RL designers of Super Mario Bros generate and concatenate level segments while
considering the diversity among the segments. The correctness of the generation
is ensured by a neural net-assisted evolutionary level repairer and the
playability of the whole level is determined through AI-based testing. Our
agents in this EDRL implementation learn to maximise a quantification of
Koster's principle of fun by moderating the degree of diversity across level
segments. Moreover, we test their ability to design fun levels that are diverse
over time and playable. Our proposed framework is capable of generating
endless, playable Super Mario Bros levels with varying degrees of fun,
deviation from earlier segments, and playability. EDRL can be generalised to
any game that is built as a segment-based sequential process and features a
built-in compressed representation of its game content.

negative:
Reinforcement learning algorithms have performed well in playing challenging
board and video games. More and more studies focus on improving the
generalisation ability of reinforcement learning algorithms. The General Video
Game AI Learning Competition aims to develop agents capable of learning to play
different game levels that were unseen during training. This paper summarises
the five years' General Video Game AI Learning Competition editions. At each
edition, three new games were designed. The training and test levels were
designed separately in the first three editions. Since 2020, three test levels
of each game were generated by perturbing or combining two training levels.
Then, we present a novel reinforcement learning technique with dual-observation
for general video game playing, assuming that it is more likely to observe
similar local information in different levels rather than global information.
Instead of directly inputting a single, raw pixel-based screenshot of the
current game screen, our proposed general technique takes the encoded,
transformed global and local observations of the game screen as two
simultaneous inputs, aiming at learning local information for playing new
levels. Our proposed technique is implemented with three state-of-the-art
reinforcement learning algorithms and tested on the game set of the 2020
General Video Game AI Learning Competition. Ablation studies show the
outstanding performance of using encoded, transformed global and local
observations as input.

negative:
Machine learning for procedural content generation has recently become an
active area of research. Levels vary in both form and function and are mostly
unrelated to each other across games. This has made it difficult to assemble
suitably large datasets to bring machine learning to level design in the same
way as it's been used for image generation. Here we propose Generative Playing
Networks which design levels for itself to play. The algorithm is built in two
parts; an agent that learns to play game levels, and a generator that learns
the distribution of playable levels. As the agent learns and improves its
ability, the space of playable levels, as defined by the agent, grows. The
generator targets the agent's playability estimates to then update its
understanding of what constitutes a playable level. We call this process of
learning the distribution of data found through self-discovery with an
environment, self-supervised inductive learning. Unlike previous approaches to
procedural content generation, Generative Playing Networks are end-to-end
differentiable and do not require human-designed examples or domain knowledge.
We demonstrate the capability of this framework by training an agent and level
generator for a 2D dungeon crawler game.

negative:
Algorithms that generate computer game content require game design knowledge.
We present an approach to automatically learn game design knowledge for level
design from gameplay videos. We further demonstrate how the acquired design
knowledge can be used to generate sections of game levels. Our approach
involves parsing video of people playing a game to detect the appearance of
patterns of sprites and utilizing machine learning to build a probabilistic
model of sprite placement. We show how rich game design information can be
automatically parsed from gameplay videos and represented as a set of
generative probabilistic models. We use Super Mario Bros. as a proof of
concept. We evaluate our approach on a measure of playability and stylistic
similarity to the original levels as represented in the gameplay videos.

negative:
This survey explores Procedural Content Generation via Machine Learning
(PCGML), defined as the generation of game content using machine learning
models trained on existing content. As the importance of PCG for game
development increases, researchers explore new avenues for generating
high-quality content with or without human involvement; this paper addresses
the relatively new paradigm of using machine learning (in contrast with
search-based, solver-based, and constructive methods). We focus on what is most
often considered functional game content such as platformer levels, game maps,
interactive fiction stories, and cards in collectible card games, as opposed to
cosmetic content such as sprites and sound effects. In addition to using PCG
for autonomous generation, co-creativity, mixed-initiative design, and
compression, PCGML is suited for repair, critique, and content analysis because
of its focus on modeling existing content. We discuss various data sources and
representations that affect the resulting generated content. Multiple PCGML
methods are covered, including neural networks, long short-term memory (LSTM)
networks, autoencoders, and deep convolutional networks; Markov models,
$n$-grams, and multi-dimensional Markov chains; clustering; and matrix
factorization. Finally, we discuss open problems in the application of PCGML,
including learning from small datasets, lack of training data, multi-layered
learning, style-transfer, parameter tuning, and PCG as a game mechanic.

negative:
Automatically adapting game content to players opens new doors for game
development. In this paper we propose an architecture using persona agents and
experience metrics, which enables evolving procedurally generated levels
tailored for particular player personas. Using our game, "Grave Rave", we
demonstrate that this approach successfully adapts to four rule-based persona
agents over three different experience metrics. Furthermore, the adaptation is
shown to be specific in nature, meaning that the levels are persona-conscious,
and not just general optimizations with regard to the selected metric.

negative:
Generative Adversarial Networks (GANs) are a powerful indirect
genotype-to-phenotype mapping for evolutionary search. Much previous work
applying GANs to level generation focuses on fixed-size segments combined into
a whole level, but individual segments may not fit together cohesively. In
contrast, segments in human designed levels are often repeated, directly or
with variation, and organized into patterns (the symmetric eagle in Level 1 of
The Legend of Zelda, or repeated pipe motifs in Super Mario Bros). Such
patterns can be produced with Compositional Pattern Producing Networks (CPPNs).
CPPNs define latent vector GAN inputs as a function of geometry, organizing
segments output by a GAN into complete levels. However, collections of latent
vectors can also be evolved directly, producing more chaotic levels. We propose
a hybrid approach that evolves CPPNs first, but allows latent vectors to evolve
later, combining the benefits of both approaches. These approaches are
evaluated in Super Mario Bros. and The Legend of Zelda. We previously
demonstrated via a Quality-Diversity algorithm that CPPNs better cover the
space of possible levels than directly evolved levels. Here, we show that the
hybrid approach (1) covers areas that neither of the other methods can, and (2)
achieves comparable or superior QD scores.

negative:
The General Video Game Artificial Intelligence (GVGAI) competition has been
running for several years with various tracks. This paper focuses on the
challenge of the GVGAI learning track in which 3 games are selected and 2
levels are given for training, while 3 hidden levels are left for evaluation.
This setup poses a difficult challenge for current Reinforcement Learning (RL)
algorithms, as they typically require much more data. This work investigates 3
versions of the Advantage Actor-Critic (A2C) algorithm trained on a maximum of
2 levels from the available 5 from the GVGAI framework and compares their
performance on all levels. The selected sub-set of games have different
characteristics, like stochasticity, reward distribution and objectives. We
found that stochasticity improves the generalisation, but too much can cause
the algorithms to fail to learn the training levels. The quality of the
training levels also matters, different sets of training levels can boost
generalisation over all levels. In the GVGAI competition agents are scored
based on their win rates and then their scores achieved in the games. We found
that solely using the rewards provided by the game might not encourage winning.

negative:
Video game level generation based on machine learning (ML), in particular,
deep generative models, has attracted attention as a technique to automate
level generation. However, applications of existing ML-based level generations
are mostly limited to tile-based level representation. When ML techniques are
applied to game domains with non-tile-based level representation, such as Angry
Birds, where objects in a level are specified by real-valued parameters, ML
often fails to generate playable levels. In this study, we develop a
deep-generative-model-based level generation for the game domain of Angry
Birds. To overcome these drawbacks, we propose a sequential encoding of a level
and process it as text data, whereas existing approaches employ a tile-based
encoding and process it as an image. Experiments show that the proposed level
generator drastically improves the stability and diversity of generated levels
compared with existing approaches. We apply latent variable evolution with the
proposed generator to control the feature of a generated level computed through
an AI agent's play, while keeping the level stable and natural.

negative:
Generative Adversarial Networks (GANs) are an emerging form of indirect
encoding. The GAN is trained to induce a latent space on training data, and a
real-valued evolutionary algorithm can search that latent space. Such Latent
Variable Evolution (LVE) has recently been applied to game levels. However, it
is hard for objective scores to capture level features that are appealing to
players. Therefore, this paper introduces a tool for interactive LVE of
tile-based levels for games. The tool also allows for direct exploration of the
latent dimensions, and allows users to play discovered levels. The tool works
for a variety of GAN models trained for both Super Mario Bros. and The Legend
of Zelda, and is easily generalizable to other games. A user study shows that
both the evolution and latent space exploration features are appreciated, with
a slight preference for direct exploration, but combining these features allows
users to discover even better levels. User feedback also indicates how this
system could eventually grow into a commercial design tool, with the addition
of a few enhancements.

negative:
Generative Adversarial Networks (GANs) are capable of generating convincing
imitations of elements from a training set, but the distribution of elements in
the training set affects to difficulty of properly training the GAN and the
quality of the outputs it produces. This paper looks at six different GANs
trained on different subsets of data from the game Lode Runner. The quality
diversity algorithm MAP-Elites was used to explore the set of quality levels
that could be produced by each GAN, where quality was defined as being beatable
and having the longest solution path possible. Interestingly, a GAN trained on
only 20 levels generated the largest set of diverse beatable levels while a GAN
trained on 150 levels generated the smallest set of diverse beatable levels,
thus challenging the notion that more is always better when training GANs.

negative:
Applying latent variable evolution to game level design has become more and
more popular as little human expert knowledge is required. However, defective
levels with illegal patterns may be generated due to the violation of
constraints for level design. A traditional way of repairing the defective
levels is programming specific rule-based repairers to patch the flaw. However,
programming these constraints is sometimes complex and not straightforward. An
autonomous level repairer which is capable of learning the constraints is
needed. In this paper, we propose a novel approach, CNet, to learn the
probability distribution of tiles giving its surrounding tiles on a set of real
levels, and then detect the illegal tiles in generated new levels. Then, an
evolutionary repairer is designed to search for optimal replacement schemes
equipped with a novel search space being constructed with the help of CNet and
a novel heuristic function. The proposed approaches are proved to be effective
in our case study of repairing GAN-generated and artificially destroyed levels
of Super Mario Bros. game. Our CNet-assisted evolutionary repairer can also be
easily applied to other games of which the levels can be represented by a
matrix of objects or tiles.

negative:
In games, as well as many user-facing systems, adapting content to users'
preferences and experience is an important challenge. This paper explores a
novel method to realize this goal in the context of dynamic difficulty
adjustment (DDA). Here the aim is to constantly adapt the content of a game to
the skill level of the player, keeping them engaged by avoiding states that are
either too difficult or too easy. Current systems for DDA rely on expensive
data mining, or on hand-crafted rules designed for particular domains, and
usually adapts to keep players in the flow, leaving no room for the designer to
present content that is purposefully easy or difficult. This paper presents
Fast Bayesian Content Adaption (FBCA), a system for DDA that is agnostic to the
domain and that can target particular difficulties. We deploy this framework in
two different domains: the puzzle game Sudoku, and a simple Roguelike game. By
modifying the acquisition function's optimization, we are reliably able to
present a content with a bespoke difficulty for players with different skill
levels in less than five iterations for Sudoku and fifteen iterations for the
simple Roguelike. Our method significantly outperforms simpler DDA heuristics
with the added benefit of maintaining a model of the user. These results point
towards a promising alternative for content adaption in a variety of different
domains.

negative:
We present practical approaches of using deep learning to create and enhance
level maps and textures for video games -- desktop, mobile, and web. We aim to
present new possibilities for game developers and level artists. The task of
designing levels and filling them with details is challenging. It is both
time-consuming and takes effort to make levels rich, complex, and with a
feeling of being natural. Fortunately, recent progress in deep learning
provides new tools to accompany level designers and visual artists. Moreover,
they offer a way to generate infinite worlds for game replayability and adjust
educational games to players' needs. We present seven approaches to create
level maps, each using statistical methods, machine learning, or deep learning.
In particular, we include:
  - Generative Adversarial Networks for creating new images from existing
examples (e.g. ProGAN).
  - Super-resolution techniques for upscaling images while preserving crisp
detail (e.g. ESRGAN).
  - Neural style transfer for changing visual themes.
  - Image translation - turning semantic maps into images (e.g. GauGAN).
  - Semantic segmentation for turning images into semantic masks (e.g. U-Net).
  - Unsupervised semantic segmentation for extracting semantic features (e.g.
Tile2Vec).
  - Texture synthesis - creating large patterns based on a smaller sample (e.g.
InGAN).

negative:
This paper presents an adaptive level generation algorithm for the
physics-based puzzle game Angry Birds. The proposed algorithm is based on a
pre-existing level generator for this game, but where the difficulty of the
generated levels can be adjusted based on the player's performance. This allows
for the creation of personalised levels tailored specifically to the player's
own abilities. The effectiveness of our proposed method is evaluated using
several agents with differing strategies and AI techniques. By using these
agents as models / representations of real human player's characteristics, we
can optimise level properties efficiently over a large number of generations.
As a secondary investigation, we also demonstrate that by combining the
performance of several agents together it is possible to generate levels that
are especially challenging for certain players but not others.

negative:
Machine playtesting tools and game moment search engines require exposure to
the diversity of a game's state space if they are to report on or index the
most interesting moments of possible play. Meanwhile, mobile app distribution
services would like to quickly determine if a freshly-uploaded game is fit to
be published. Having access to a semantic map of reachable states in the game
would enable efficient inference in these applications. However, human gameplay
data is expensive to acquire relative to the coverage of a game that it
provides. We show that off-the-shelf automatic exploration strategies can
explore with an effectiveness comparable to human gameplay on the same
timescale. We contribute generic methods for quantifying exploration quality as
a function of time and demonstrate our metric on several elementary techniques
and human players on a collection of commercial games sampled from multiple
game platforms (from Atari 2600 to Nintendo 64). Emphasizing the diversity of
states reached and the semantic map extracted, this work makes productive
contrast with the focus on finding a behavior policy or optimizing game score
used in most automatic game playing research.

negative:
This paper provides a detailed investigation of using the Kullback-Leibler
(KL) Divergence as a way to compare and analyse game-levels, and hence to use
the measure as the objective function of an evolutionary algorithm to evolve
new levels. We describe the benefits of its asymmetry for level analysis and
demonstrate how (not surprisingly) the quality of the results depends on the
features used. Here we use tile-patterns of various sizes as features.
  When using the measure for evolution-based level generation, we demonstrate
that the choice of variation operator is critical in order to provide an
efficient search process, and introduce a novel convolutional mutation operator
to facilitate this. We compare the results with alternative generators,
including evolving in the latent space of generative adversarial networks, and
Wave Function Collapse. The results clearly show the proposed method to provide
competitive performance, providing reasonable quality results with very fast
training and reasonably fast generation.

negative:
The Angry Birds AI competition has been held over many years to encourage the
development of AI agents that can play Angry Birds game levels better than
human players. Many different agents with various approaches have been employed
over the competition's lifetime to solve this task. Even though the performance
of these agents has increased significantly over the past few years, they still
show major drawbacks in playing deceptive levels. This is because most of the
current agents try to identify the best next shot rather than planning an
effective sequence of shots. In order to encourage advancements in such agents,
we present an automated methodology to generate deceptive game levels for Angry
Birds. Even though there are many existing content generators for Angry Birds,
they do not focus on generating deceptive levels. In this paper, we propose a
procedure to generate deceptive levels for six deception categories that can
fool the state-of-the-art Angry Birds playing AI agents. Our results show that
generated deceptive levels exhibit similar characteristics of human-created
deceptive levels. Additionally, we define metrics to measure the stability,
solvability, and degree of deception of the generated levels.

negative:
Generating novel graph structures that optimize given objectives while
obeying some given underlying rules is fundamental for chemistry, biology and
social science research. This is especially important in the task of molecular
graph generation, whose goal is to discover novel molecules with desired
properties such as drug-likeness and synthetic accessibility, while obeying
physical laws such as chemical valency. However, designing models to find
molecules that optimize desired properties while incorporating highly complex
and non-differentiable rules remains to be a challenging task. Here we propose
Graph Convolutional Policy Network (GCPN), a general graph convolutional
network based model for goal-directed graph generation through reinforcement
learning. The model is trained to optimize domain-specific rewards and
adversarial loss through policy gradient, and acts in an environment that
incorporates domain-specific rules. Experimental results show that GCPN can
achieve 61% improvement on chemical property optimization over state-of-the-art
baselines while resembling known molecules, and achieve 184% improvement on the
constrained property optimization task.

negative:
Recent procedural content generation via machine learning (PCGML) methods
allow learning from existing content to produce similar content automatically.
While these approaches are able to generate content for different games (e.g.
Super Mario Bros., DOOM, Zelda, and Kid Icarus), it is an open questions how
well these approaches can capture large-scale visual patterns such as symmetry.
In this paper, we propose match-three games as a domain to test PCGML
algorithms regarding their ability to generate suitable patterns. We
demonstrate that popular algorithm such as Generative Adversarial Networks
struggle in this domain and propose adaptations to improve their performance.
In particular we augment the neighborhood of a Markov Random Fields approach to
not only take local but also symmetric positional information into account. We
conduct several empirical tests including a user study that show the
improvements achieved by the proposed modifications, and obtain promising
results.

negative:
We propose the problem of tutorial generation for games, i.e. to generate
tutorials which can teach players to play games, as an AI problem. This problem
can be approached in several ways, including generating natural language
descriptions of game rules, generating instructive game levels, and generating
demonstrations of how to play a game using agents that play in a human-like
manner. We further argue that the General Video Game AI framework provides a
useful testbed for addressing this problem.

negative:
Generative Adversarial Networks (GANs) can generate levels for a variety of
games. This paper focuses on combining GAN-generated segments in a snaking
pattern to create levels for Mega Man. Adjacent segments in such levels can be
orthogonally adjacent in any direction, meaning that an otherwise fine segment
might impose a barrier between its neighbor depending on what sorts of segments
in the training set are being most closely emulated: horizontal, vertical, or
corner segments. To pick appropriate segments, multiple GANs were trained on
different types of segments to ensure better flow between segments. Flow was
further improved by evolving the latent vectors for the segments being joined
in the level to maximize the length of the level's solution path. Using
multiple GANs to represent different types of segments results in significantly
longer solution paths than using one GAN for all segment types, and a human
subject study verifies that these levels are more fun and have more human-like
design than levels produced by one GAN.

negative:
Successfully navigating a complex environment to obtain a desired outcome is
a difficult task, that up to recently was believed to be capable only by
humans. This perception has been broken down over time, especially with the
introduction of deep reinforcement learning, which has greatly increased the
difficulty of tasks that can be automated. However, for traditional
reinforcement learning agents this requires an environment to be able to
provide frequent extrinsic rewards, which are not known or accessible for many
real-world environments. This project aims to explore and contrast existing
reinforcement learning solutions that circumnavigate the difficulties of an
environment that provide sparse rewards. Different reinforcement solutions will
be implemented over a several video game environments with varying difficulty
and varying frequency of rewards, as to properly investigate the applicability
of these solutions. This project introduces a novel reinforcement learning
solution by combining aspects of two existing state of the art sparse reward
solutions, curiosity driven exploration and unsupervised auxiliary tasks.

negative:
Levels are a key component of many different video games, and a large body of
work has been produced on how to procedurally generate game levels. Recently,
Machine Learning techniques have been applied to video game level generation
towards the purpose of automatically generating levels that have the properties
of the training corpus. Towards that end we have made available a corpora of
video game levels in an easy to parse format ideal for different machine
learning and other game AI research purposes.

negative:
We describe a search-based approach to generating new levels for bullet hell
games, which are action games characterized by and requiring avoidance of a
very large amount of projectiles. Levels are represented using a
domain-specific description language, and search in the space defined by this
language is performed by a novel variant of the Map-Elites algorithm which
incorporates a feasible- infeasible approach to constraint satisfaction.
Simulation-based evaluation is used to gauge the fitness of levels, using an
agent based on best-first search. The performance of the agent can be tuned
according to the two dimensions of strategy and dexterity, making it possible
to search for level configurations that require a specific combination of both.
As far as we know, this paper describes the first generator for this game
genre, and includes several algorithmic innovations.

negative:
This paper reviews the field of Game AI, which not only deals with creating
agents that can play a certain game, but also with areas as diverse as creating
game content automatically, game analytics, or player modelling. While Game AI
was for a long time not very well recognized by the larger scientific
community, it has established itself as a research area for developing and
testing the most advanced forms of AI algorithms and articles covering advances
in mastering video games such as StarCraft 2 and Quake III appear in the most
prestigious journals. Because of the growth of the field, a single review
cannot cover it completely. Therefore, we put a focus on important recent
developments, including that advances in Game AI are starting to be extended to
areas outside of games, such as robotics or the synthesis of chemicals. In this
article, we review the algorithms and methods that have paved the way for these
breakthroughs, report on the other important areas of Game AI research, and
also point out exciting directions for the future of Game AI.

negative:
We introduce the General Video Game Rule Generation problem, and the
eponymous software framework which will be used in a new track of the General
Video Game AI (GVGAI) competition. The problem is, given a game level as input,
to generate the rules of a game that fits that level. This can be seen as the
inverse of the General Video Game Level Generation problem. Conceptualizing
these two problems as separate helps breaking the very hard problem of
generating complete games into smaller, more manageable subproblems. The
proposed framework builds on the GVGAI software and thus asks the rule
generator for rules defined in the Video Game Description Language. We describe
the API, and three different rule generators: a random, a constructive and a
search-based generator. Early results indicate that the constructive generator
generates playable and somewhat interesting game rules but has a limited
expressive range, whereas the search-based generator generates remarkably
diverse rulesets, but with an uneven quality.

negative:
Automated game design is the problem of automatically producing games through
computational processes. Traditionally, these methods have relied on the
authoring of search spaces by a designer, defining the space of all possible
games for the system to author. In this paper, we instead learn representations
of existing games from gameplay video and use these to approximate a search
space of novel games. In a human subject study we demonstrate that these novel
games are indistinguishable from human games in terms of challenge, and that
one of the novel games was equivalent to one of the human games in terms of
fun, frustration, and likeability.

negative:
Modern machine learning techniques, such as deep neural networks, are
transforming many disciplines ranging from image recognition to language
understanding, by uncovering patterns in big data and making accurate
predictions. They have also shown promising results for synthesizing new
designs, which is crucial for creating products and enabling innovation.
Generative models, including generative adversarial networks (GANs), have
proven to be effective for design synthesis with applications ranging from
product design to metamaterial design. These automated computational design
methods can support human designers, who typically create designs by a
time-consuming process of iteratively exploring ideas using experience and
heuristics. However, there are still challenges remaining in automatically
synthesizing `creative' designs. GAN models, however, are not capable of
generating unique designs, a key to innovation and a major gap in AI-based
design automation applications. This paper proposes an automated method, named
CreativeGAN, for generating novel designs. It does so by identifying components
that make a design unique and modifying a GAN model such that it becomes more
likely to generate designs with identified unique components. The method
combines state-of-art novelty detection, segmentation, novelty localization,
rewriting, and generative models for creative design synthesis. Using a dataset
of bicycle designs, we demonstrate that the method can create new bicycle
designs with unique frames and handles, and generalize rare novelties to a
broad set of designs. Our automated method requires no human intervention and
demonstrates a way to rethink creative design synthesis and exploration.

negative:
Bayesian optimization has been successful at global optimization of
expensive-to-evaluate multimodal objective functions. However, unlike most
optimization methods, Bayesian optimization typically does not use derivative
information. In this paper we show how Bayesian optimization can exploit
derivative information to decrease the number of objective function evaluations
required for good performance. In particular, we develop a novel Bayesian
optimization algorithm, the derivative-enabled knowledge-gradient (dKG), for
which we show one-step Bayes-optimality, asymptotic consistency, and greater
one-step value of information than is possible in the derivative-free setting.
Our procedure accommodates noisy and incomplete derivative information, comes
in both sequential and batch forms, and can optionally reduce the computational
cost of inference through automatically selected retention of a single
directional derivative. We also compute the d-KG acquisition function and its
gradient using a novel fast discretization-free technique. We show d-KG
provides state-of-the-art performance compared to a wide range of optimization
procedures with and without gradients, on benchmarks including logistic
regression, deep learning, kernel learning, and k-nearest neighbors.

negative:
The procedural generation of video game levels has existed for at least 30
years, but only recently have machine learning approaches been used to generate
levels without specifying the rules for generation. A number of these have
looked at platformer levels as a sequence of characters and performed
generation using Markov chains. In this paper we examine the use of Long
Short-Term Memory recurrent neural networks (LSTMs) for the purpose of
generating levels trained from a corpus of Super Mario Brothers levels. We
analyze a number of different data representations and how the generated levels
fit into the space of human authored Super Mario Brothers levels.

negative:
Multiple Artificial Intelligence (AI) methods have been proposed over recent
years to create controllers to play multiple video games of different nature
and complexity without revealing the specific mechanics of each of these games
to the AI methods. In recent years, Evolutionary Algorithms (EAs) employing
rolling horizon mechanisms have achieved extraordinary results in these type of
problems. However, some limitations are present in Rolling Horizon EAs making
it a grand challenge of AI. These limitations include the wasteful mechanism of
creating a population and evolving it over a fraction of a second to propose an
action to be executed by the game agent. Another limitation is to use a scalar
value (fitness value) to direct evolutionary search instead of accounting for a
mechanism that informs us how a particular agent behaves during the rolling
horizon simulation. In this work, we address both of these issues. We introduce
the use of a statistical tree that tackles the latter limitation. Furthermore,
we tackle the former limitation by employing a mechanism that allows us to seed
part of the population using Monte Carlo Tree Search, a method that has
dominated multiple General Video Game AI competitions. We show how the proposed
novel mechanism, called Statistical Tree-based Population Seeding, achieves
better results compared to vanilla Rolling Horizon EAs in a set of 20 games,
including 10 stochastic and 10 deterministic games.

negative:
Procedural content generation via machine learning (PCGML) has shown success
at producing new video game content with machine learning. However, the
majority of the work has focused on the production of static game content,
including game levels and visual elements. There has been much less work on
dynamic game content, such as game mechanics. One reason for this is the lack
of a consistent representation for dynamic game content, which is key for a
number of statistical machine learning approaches. We present an autoencoder
for deriving what we call "entity embeddings", a consistent way to represent
different dynamic entities across multiple games in the same representation. In
this paper we introduce the learned representation, along with some evidence
towards its quality and future utility.

negative:
This work introduces World-GAN, the first method to perform data-driven
Procedural Content Generation via Machine Learning in Minecraft from a single
example. Based on a 3D Generative Adversarial Network (GAN) architecture, we
are able to create arbitrarily sized world snippets from a given sample. We
evaluate our approach on creations from the community as well as structures
generated with the Minecraft World Generator. Our method is motivated by the
dense representations used in Natural Language Processing (NLP) introduced with
word2vec [1]. The proposed block2vec representations make World-GAN independent
from the number of different blocks, which can vary a lot in Minecraft, and
enable the generation of larger levels. Finally, we demonstrate that changing
this new representation space allows us to change the generated style of an
already trained generator. World-GAN enables its users to generate Minecraft
worlds based on parts of their creations.

negative:
Access to a fast and easily copied forward model of a game is essential for
model-based reinforcement learning and for algorithms such as Monte Carlo tree
search, and is also beneficial as a source of unlimited experience data for
model-free algorithms. Learning forward models is an interesting and important
challenge in order to address problems where a model is not available. Building
upon previous work on the Neural GPU, this paper introduces the Neural Game
Engine, as a way to learn models directly from pixels. The learned models are
able to generalise to different size game levels to the ones they were trained
on without loss of accuracy. Results on 10 deterministic General Video Game AI
games demonstrate competitive performance, with many of the games models being
learned perfectly both in terms of pixel predictions and reward predictions.
The pre-trained models are available through the OpenAI Gym interface and are
available publicly for future research here:
\url{https://github.com/Bam4d/Neural-Game-Engine}

negative:
In this paper we present a novel approach to optimise tactical and strategic
decision making in football (soccer). We model the game of football as a
multi-stage game which is made up from a Bayesian game to model the pre-match
decisions and a stochastic game to model the in-match state transitions and
decisions. Using this formulation, we propose a method to predict the
probability of game outcomes and the payoffs of team actions. Building upon
this, we develop algorithms to optimise team formation and in-game tactics with
different objectives. Empirical evaluation of our approach on real-world
datasets from 760 matches shows that by using optimised tactics from our
Bayesian and stochastic games, we can increase a team chances of winning by up
to 16.1\% and 3.4\% respectively.

negative:
The design of video game levels is a complex and critical task. Levels need
to elicit fun and challenge while avoiding frustration at all costs. In this
paper, we present a framework to assist designers in the creation of levels for
2D platformers. Our framework provides designers with a toolbox (i) to create
2D platformer levels, (ii) to estimate the difficulty and probability of
success of single jump actions (the main mechanics of platformer games), and
(iii) a set of metrics to evaluate the difficulty and probability of completion
of entire levels. At the end, we present the results of a set of experiments we
carried out with human players to validate the metrics included in our
framework.

negative:
Environments with procedurally generated content serve as important
benchmarks for testing systematic generalization in deep reinforcement
learning. In this setting, each level is an algorithmically created environment
instance with a unique configuration of its factors of variation. Training on a
prespecified subset of levels allows for testing generalization to unseen
levels. What can be learned from a level depends on the current policy, yet
prior work defaults to uniform sampling of training levels independently of the
policy. We introduce Prioritized Level Replay (PLR), a general framework for
selectively sampling the next training level by prioritizing those with higher
estimated learning potential when revisited in the future. We show TD-errors
effectively estimate a level's future learning potential and, when used to
guide the sampling procedure, induce an emergent curriculum of increasingly
difficult levels. By adapting the sampling of training levels, PLR
significantly improves sample efficiency and generalization on Procgen
Benchmark--matching the previous state-of-the-art in test return--and readily
combines with other methods. Combined with the previous leading method, PLR
raises the state-of-the-art to over 76% improvement in test return relative to
standard RL baselines.

negative:
Reinforcement learning (RL) studies how an agent comes to achieve reward in
an environment through interactions over time. Recent advances in machine RL
have surpassed human expertise at the world's oldest board games and many
classic video games, but they require vast quantities of experience to learn
successfully -- none of today's algorithms account for the human ability to
learn so many different tasks, so quickly. Here we propose a new approach to
this challenge based on a particularly strong form of model-based RL which we
call Theory-Based Reinforcement Learning, because it uses human-like intuitive
theories -- rich, abstract, causal models of physical objects, intentional
agents, and their interactions -- to explore and model an environment, and plan
effectively to achieve task goals. We instantiate the approach in a video game
playing agent called EMPA (the Exploring, Modeling, and Planning Agent), which
performs Bayesian inference to learn probabilistic generative models expressed
as programs for a game-engine simulator, and runs internal simulations over
these models to support efficient object-based, relational exploration and
heuristic planning. EMPA closely matches human learning efficiency on a suite
of 90 challenging Atari-style video games, learning new games in just minutes
of game play and generalizing robustly to new game situations and new levels.
The model also captures fine-grained structure in people's exploration
trajectories and learning dynamics. Its design and behavior suggest a way
forward for building more general human-like AI systems.

negative:
General Video Game Playing (GVGP) aims at designing an agent that is capable
of playing multiple video games with no human intervention. In 2014, The
General Video Game AI (GVGAI) competition framework was created and released
with the purpose of providing researchers a common open-source and easy to use
platform for testing their AI methods with potentially infinity of games
created using Video Game Description Language (VGDL). The framework has been
expanded into several tracks during the last few years to meet the demand of
different research directions. The agents are required either to play multiple
unknown games with or without access to game simulations, or to design new game
levels or rules. This survey paper presents the VGDL, the GVGAI framework,
existing tracks, and reviews the wide use of GVGAI framework in research,
education and competitions five years after its birth. A future plan of
framework improvements is also described.

negative:
Generative Adversarial Networks (GANs) have recently attracted considerable
attention in the AI community due to its ability to generate high-quality data
of significant statistical resemblance to real data. Fundamentally, GAN is a
game between two neural networks trained in an adversarial manner to reach a
zero-sum Nash equilibrium profile. Despite the improvement accomplished in GANs
in the last few years, several issues remain to be solved. This paper reviews
the literature on the game theoretic aspects of GANs and addresses how game
theory models can address specific challenges of generative model and improve
the GAN's performance. We first present some preliminaries, including the basic
GAN model and some game theory background. We then present taxonomy to classify
state-of-the-art solutions into three main categories: modified game models,
modified architectures, and modified learning methods. The classification is
based on modifications made to the basic GAN model by proposed game-theoretic
approaches in the literature. We then explore the objectives of each category
and discuss recent works in each category. Finally, we discuss the remaining
challenges in this field and present future research directions.

negative:
Procedural Content Generation via Machine Learning (PCGML) refers to a group
of methods for creating game content (e.g. platformer levels, game maps, etc.)
using machine learning models. PCGML approaches rely on black box models, which
can be difficult to understand and debug by human designers who do not have
expert knowledge about machine learning. This can be even more tricky in
co-creative systems where human designers must interact with AI agents to
generate game content. In this paper we present an approach to explainable
artificial intelligence in which certain training instances are offered to
human users as an explanation for the AI agent's actions during a co-creation
process. We evaluate this approach by approximating its ability to provide
human users with the explanations of AI agent's actions and helping them to
more efficiently cooperate with the AI agent.

negative:
Generative models are undoubtedly a hot topic in Artificial Intelligence,
among which the most common type is Generative Adversarial Networks (GANs).
These architectures let one synthesise artificial datasets by implicitly
modelling the underlying probability distribution of a real-world training
dataset. With the introduction of Conditional GANs and their variants, these
methods were extended to generating samples conditioned on ancillary
information available for each sample within the dataset. From a practical
standpoint, however, one might desire to generate data conditioned on partial
information. That is, only a subset of the ancillary conditioning variables
might be of interest when synthesising data. In this work, we argue that
standard Conditional GANs are not suitable for such a task and propose a new
Adversarial Network architecture and training strategy to deal with the ensuing
problems. Experiments illustrating the value of the proposed approach in digit
and face image synthesis under partial conditioning information are presented,
showing that the proposed method can effectively outperform the standard
approach under these circumstances.

negative:
The task of image generation started to receive some attention from artists
and designers to inspire them in new creations. However, exploiting the results
of deep generative models such as Generative Adversarial Networks can be long
and tedious given the lack of existing tools. In this work, we propose a simple
strategy to inspire creators with new generations learned from a dataset of
their choice, while providing some control on them. We design a simple
optimization method to find the optimal latent parameters corresponding to the
closest generation to any input inspirational image. Specifically, we allow the
generation given an inspirational image of the user choice by performing
several optimization steps to recover optimal parameters from the model's
latent space. We tested several exploration methods starting with classic
gradient descents to gradient-free optimizers. Many gradient-free optimizers
just need comparisons (better/worse than another image), so that they can even
be used without numerical criterion, without inspirational image, but with only
with human preference. Thus, by iterating on one's preferences we could make
robust Facial Composite or Fashion Generation algorithms. High resolution of
the produced design generations are obtained using progressive growing of GANs.
Our results on four datasets of faces, fashion images, and textures show that
satisfactory images are effectively retrieved in most cases.

negative:
As a new way of training generative models, Generative Adversarial Nets (GAN)
that uses a discriminative model to guide the training of the generative model
has enjoyed considerable success in generating real-valued data. However, it
has limitations when the goal is for generating sequences of discrete tokens. A
major reason lies in that the discrete outputs from the generative model make
it difficult to pass the gradient update from the discriminative model to the
generative model. Also, the discriminative model can only assess a complete
sequence, while for a partially generated sequence, it is non-trivial to
balance its current score and the future one once the entire sequence has been
generated. In this paper, we propose a sequence generation framework, called
SeqGAN, to solve the problems. Modeling the data generator as a stochastic
policy in reinforcement learning (RL), SeqGAN bypasses the generator
differentiation problem by directly performing gradient policy update. The RL
reward signal comes from the GAN discriminator judged on a complete sequence,
and is passed back to the intermediate state-action steps using Monte Carlo
search. Extensive experiments on synthetic data and real-world tasks
demonstrate significant improvements over strong baselines.

negative:
Despite recent advances, goal-directed generation of structured discrete data
remains challenging. For problems such as program synthesis (generating source
code) and materials design (generating molecules), finding examples which
satisfy desired constraints or exhibit desired properties is difficult. In
practice, expensive heuristic search or reinforcement learning algorithms are
often employed. In this paper we investigate the use of conditional generative
models which directly attack this inverse problem, by modeling the distribution
of discrete structures given properties of interest. Unfortunately, maximum
likelihood training of such models often fails with the samples from the
generative model inadequately respecting the input properties. To address this,
we introduce a novel approach to directly optimize a reinforcement learning
objective, maximizing an expected reward. We avoid high-variance score-function
estimators that would otherwise be required by sampling from an approximation
to the normalized rewards, allowing simple Monte Carlo estimation of model
gradients. We test our methodology on two tasks: generating molecules with
user-defined properties and identifying short python expressions which evaluate
to a given target value. In both cases, we find improvements over maximum
likelihood estimation and other baselines.

negative:
Angry Birds is a popular video game in which the player is provided with a
sequence of birds to shoot from a slingshot. The task of the game is to destroy
all green pigs with maximum possible score. Angry Birds appears to be a
difficult task to solve for artificially intelligent agents due to the
sequential decision-making, non-deterministic game environment, enormous state
and action spaces and requirement to differentiate between multiple birds,
their abilities and optimum tapping times. We describe the application of Deep
Reinforcement learning by implementing Double Dueling Deep Q-network to play
Angry Birds game. One of our main goals was to build an agent that is able to
compete with previous participants and humans on the first 21 levels. In order
to do so, we have collected a dataset of game frames that we used to train our
agent on. We present different approaches and settings for DQN agent. We
evaluate our agent using results of the previous participants of AIBirds
competition, results of volunteer human players and present the results of
AIBirds 2018 competition.

negative:
Multiplayer games have long been used as testbeds in artificial intelligence
research, aptly referred to as the Drosophila of artificial intelligence.
Traditionally, researchers have focused on using well-known games to build
strong agents. This progress, however, can be better informed by characterizing
games and their topological landscape. Tackling this latter question can
facilitate understanding of agents and help determine what game an agent should
target next as part of its training. Here, we show how network measures applied
to response graphs of large-scale games enable the creation of a landscape of
games, quantifying relationships between games of varying sizes and
characteristics. We illustrate our findings in domains ranging from canonical
games to complex empirical games capturing the performance of trained agents
pitted against one another. Our results culminate in a demonstration leveraging
this information to generate new and interesting games, including mixtures of
empirical games synthesized from real world games.

negative:
Domain knowledge is crucial for effective performance in autonomous control
systems. Typically, human effort is required to encode this knowledge into a
control algorithm. In this paper, we present an approach to language grounding
which automatically interprets text in the context of a complex control
application, such as a game, and uses domain knowledge extracted from the text
to improve control performance. Both text analysis and control strategies are
learned jointly using only a feedback signal inherent to the application. To
effectively leverage textual information, our method automatically extracts the
text segment most relevant to the current game state, and labels it with a
task-centric predicate structure. This labeled text is then used to bias an
action selection policy for the game, guiding it towards promising regions of
the action space. We encode our model for text analysis and game playing in a
multi-layer neural network, representing linguistic decisions via latent
variables in the hidden layers, and game action quality via the output layer.
Operating within the Monte-Carlo Search framework, we estimate model parameters
using feedback from simulated games. We apply our approach to the complex
strategy game Civilization II using the official game manual as the text guide.
Our results show that a linguistically-informed game-playing agent
significantly outperforms its language-unaware counterpart, yielding a 34%
absolute improvement and winning over 65% of games when playing against the
built-in AI of Civilization.

negative:
This paper introduces a fully automatic method for generating video game
tutorials. The AtDELFI system (AuTomatically DEsigning Legible, Full
Instructions for games) was created to investigate procedural generation of
instructions that teach players how to play video games. We present a
representation of game rules and mechanics using a graph system as well as a
tutorial generation method that uses said graph representation. We demonstrate
the concept by testing it on games within the General Video Game Artificial
Intelligence (GVG-AI) framework; the paper discusses tutorials generated for
eight different games. Our findings suggest that a graph representation scheme
works well for simple arcade style games such as Space Invaders and Pacman, but
it appears that tutorials for more complex games might require higher-level
understanding of the game than just single mechanics.

negative:
Policy gradients-based reinforcement learning has proven to be a promising
approach for directly optimizing non-differentiable evaluation metrics for
language generation tasks. However, optimizing for a specific metric reward
leads to improvements in mostly that metric only, suggesting that the model is
gaming the formulation of that metric in a particular way without often
achieving real qualitative improvements. Hence, it is more beneficial to make
the model optimize multiple diverse metric rewards jointly. While appealing,
this is challenging because one needs to manually decide the importance and
scaling weights of these metric rewards. Further, it is important to consider
using a dynamic combination and curriculum of metric rewards that flexibly
changes over time. Considering the above aspects, in our work, we automate the
optimization of multiple metric rewards simultaneously via a multi-armed bandit
approach (DORB), where at each round, the bandit chooses which metric reward to
optimize next, based on expected arm gains. We use the Exp3 algorithm for
bandits and formulate two approaches for bandit rewards: (1) Single
Multi-reward Bandit (SM-Bandit); (2) Hierarchical Multi-reward Bandit
(HM-Bandit). We empirically show the effectiveness of our approaches via
various automatic metrics and human evaluation on two important NLG tasks:
question generation and data-to-text generation, including on an unseen-test
transfer setup. Finally, we present interpretable analyses of the learned
bandit curriculum over the optimized rewards.

negative:
Behavioural cloning, where a computer is taught to perform a task based on
demonstrations, has been successfully applied to various video games and
robotics tasks, with and without reinforcement learning. This also includes
end-to-end approaches, where a computer plays a video game like humans do: by
looking at the image displayed on the screen, and sending keystrokes to the
game. As a general approach to playing video games, this has many inviting
properties: no need for specialized modifications to the game, no lengthy
training sessions and the ability to re-use the same tools across different
games. However, related work includes game-specific engineering to achieve the
results. We take a step towards a general approach and study the general
applicability of behavioural cloning on twelve video games, including six
modern video games (published after 2010), by using human demonstrations as
training data. Our results show that these agents cannot match humans in raw
performance but do learn basic dynamics and rules. We also demonstrate how the
quality of the data matters, and how recording data from humans is subject to a
state-action mismatch, due to human reflexes.

negative:
General Video Game Artificial Intelligence is a general game playing
framework for Artificial General Intelligence research in the video-games
domain. In this paper, we propose for the first time a screen capture learning
agent for General Video Game AI framework. A Deep Q-Network algorithm was
applied and improved to develop an agent capable of learning to play different
games in the framework. After testing this algorithm using various games of
different categories and difficulty levels, the results suggest that our
proposed screen capture learning agent has the potential to learn many
different games using only a single learning algorithm.

negative:
Generative Adversarial Networks (GANs) are formulated as minimax game
problems, whereby generators attempt to approach real data distributions by
virtue of adversarial learning against discriminators. The intrinsic problem
complexity poses the challenge to enhance the performance of generative
networks. In this work, we aim to boost model learning from the perspective of
network architectures, by incorporating recent progress on automated
architecture search into GANs. To this end, we propose a fully differentiable
search framework for generative adversarial networks, dubbed alphaGAN. The
searching process is formalized as solving a bi-level minimax optimization
problem, in which the outer-level objective aims for seeking a suitable network
architecture towards pure Nash Equilibrium conditioned on the generator and the
discriminator network parameters optimized with a traditional GAN loss in the
inner level. The entire optimization performs a first-order method by
alternately minimizing the two-level objective in a fully differentiable
manner, enabling architecture search to be completed in an enormous search
space. Extensive experiments on CIFAR-10 and STL-10 datasets show that our
algorithm can obtain high-performing architectures only with 3-GPU hours on a
single GPU in the search space comprised of approximate 2 ? 1011 possible
configurations. We also provide a comprehensive analysis on the behavior of the
searching process and the properties of searched architectures, which would
benefit further research on architectures for generative models. Pretrained
models and codes are available at https://github.com/yuesongtian/AlphaGAN.

negative:
Applying generative adversarial networks (GANs) to text-related tasks is
challenging due to the discrete nature of language. One line of research
resolves this issue by employing reinforcement learning (RL) and optimizing the
next-word sampling policy directly in a discrete action space. Such methods
compute the rewards from complete sentences and avoid error accumulation due to
exposure bias. Other approaches employ approximation techniques that map the
text to continuous representation in order to circumvent the non-differentiable
discrete process. Particularly, autoencoder-based methods effectively produce
robust representations that can model complex discrete structures. In this
paper, we propose a novel text GAN architecture that promotes the collaborative
training of the continuous-space and discrete-space methods. Our method employs
an autoencoder to learn an implicit data manifold, providing a learning
objective for adversarial training in a continuous space. Furthermore, the
complete textual output is directly evaluated and updated via RL in a discrete
space. The collaborative interplay between the two adversarial trainings
effectively regularize the text representations in different spaces. The
experimental results on three standard benchmark datasets show that our model
substantially outperforms state-of-the-art text GANs with respect to quality,
diversity, and global consistency.

negative:
Open-domain dialog generation is a challenging problem; maximum likelihood
training can lead to repetitive outputs, models have difficulty tracking
long-term conversational goals, and training on standard movie or online
datasets may lead to the generation of inappropriate, biased, or offensive
text. Reinforcement Learning (RL) is a powerful framework that could
potentially address these issues, for example by allowing a dialog model to
optimize for reducing toxicity and repetitiveness. However, previous approaches
which apply RL to open-domain dialog generation do so at the word level, making
it difficult for the model to learn proper credit assignment for long-term
conversational rewards. In this paper, we propose a novel approach to
hierarchical reinforcement learning, VHRL, which uses policy gradients to tune
the utterance-level embedding of a variational sequence model. This
hierarchical approach provides greater flexibility for learning long-term,
conversational rewards. We use self-play and RL to optimize for a set of
human-centered conversation metrics, and show that our approach provides
significant improvements -- in terms of both human evaluation and automatic
metrics -- over state-of-the-art dialog models, including Transformers.

negative:
Representing games through their pixels offers a promising approach for
building general-purpose and versatile game models. While games are not merely
images, neural network models trained on game pixels often capture differences
of the visual style of the image rather than the content of the game. As a
result, such models cannot generalize well even within similar games of the
same genre. In this paper we build on recent advances in contrastive learning
and showcase its benefits for representation learning in games. Learning to
contrast images of games not only classifies games in a more efficient manner;
it also yields models that separate games in a more meaningful fashion by
ignoring the visual style and focusing, instead, on their content. Our results
in a large dataset of sports video games containing 100k images across 175
games and 10 game genres suggest that contrastive learning is better suited for
learning generalized game representations compared to conventional supervised
learning. The findings of this study bring us closer to universal visual
encoders for games that can be reused across previously unseen games without
requiring retraining or fine-tuning.

negative:
Fuzzing is the process of finding security vulnerabilities in
input-processing code by repeatedly testing the code with modified inputs. In
this paper, we formalize fuzzing as a reinforcement learning problem using the
concept of Markov decision processes. This in turn allows us to apply
state-of-the-art deep Q-learning algorithms that optimize rewards, which we
define from runtime properties of the program under test. By observing the
rewards caused by mutating with a specific set of actions performed on an
initial program input, the fuzzing agent learns a policy that can next generate
new higher-reward inputs. We have implemented this new approach, and
preliminary empirical evidence shows that reinforcement fuzzing can outperform
baseline random fuzzing.

negative:
The process of playtesting a game is subjective, expensive and incomplete. In
this paper, we present a playtesting approach that explores the game space with
automated agents and collects data to answer questions posed by the designers.
Rather than have agents interacting with an actual game client, this approach
recreates the bare bone mechanics of the game as a separate system. Our agent
is able to play in minutes what would take testers days of organic gameplay.
The analysis of thousands of game simulations exposed imbalances in game
actions, identified inconsequential rewards and evaluated the effectiveness of
optional strategic choices. Our test case game, The Sims Mobile, was recently
released and the findings shown here influenced design changes that resulted in
improved player experience.

negative:
A neuroscience method to understanding the brain is to find and study the
preferred stimuli that highly activate an individual cell or groups of cells.
Recent advances in machine learning enable a family of methods to synthesize
preferred stimuli that cause a neuron in an artificial or biological brain to
fire strongly. Those methods are known as Activation Maximization (AM) or
Feature Visualization via Optimization. In this chapter, we (1) review existing
AM techniques in the literature; (2) discuss a probabilistic interpretation for
AM; and (3) review the applications of AM in debugging and explaining networks.

negative:
The automatic generation of game tutorials is a challenging AI problem. While
it is possible to generate annotations and instructions that explain to the
player how the game is played, this paper focuses on generating a gameplay
experience that introduces the player to a game mechanic. It evolves small
levels for the Mario AI Framework that can only be beaten by an agent that
knows how to perform specific actions in the game. It uses variations of a
perfect A* agent that are limited in various ways, such as not being able to
jump high or see enemies, to test how failing to do certain actions can stop
the player from beating the level.

negative:
In this article, we review recent Deep Learning advances in the context of
how they have been applied to play different types of video games such as
first-person shooters, arcade games, and real-time strategy games. We analyze
the unique requirements that different game genres pose to a deep learning
system and highlight important open challenges in the context of applying these
machine learning methods to video games, such as general game playing, dealing
with extremely large decision spaces and sparse rewards.

negative:
While Generative Adversarial Networks (GANs) show increasing performance and
the level of realism is becoming indistinguishable from natural images, this
also comes with high demands on data and computation. We show that
state-of-the-art GAN models -- such as they are being publicly released by
researchers and industry -- can be used for a range of applications beyond
unconditional image generation. We achieve this by an iterative scheme that
also allows gaining control over the image generation process despite the
highly non-linear latent spaces of the latest GAN models. We demonstrate that
this opens up the possibility to re-use state-of-the-art, difficult to train,
pre-trained GANs with a high level of control even if only black-box access is
granted. Our work also raises concerns and awareness that the use cases of a
published GAN model may well reach beyond the creators' intention, which needs
to be taken into account before a full public release. Code is available at
https://github.com/a514514772/hijackgan.

negative:
Generative Adversarial Networks (GANs) have been extremely successful in
various application domains such as computer vision, medicine, and natural
language processing. Moreover, transforming an object or person to a desired
shape become a well-studied research in the GANs. GANs are powerful models for
learning complex distributions to synthesize semantically meaningful samples.
However, there is a lack of comprehensive review in this field, especially lack
of a collection of GANs loss-variant, evaluation metrics, remedies for diverse
image generation, and stable training. Given the current fast GANs development,
in this survey, we provide a comprehensive review of adversarial models for
image synthesis. We summarize the synthetic image generation methods, and
discuss the categories including image-to-image translation, fusion image
generation, label-to-image mapping, and text-to-image translation. We organize
the literature based on their base models, developed ideas related to
architectures, constraints, loss functions, evaluation metrics, and training
datasets. We present milestones of adversarial models, review an extensive
selection of previous works in various categories, and present insights on the
development route from the model-based to data-driven methods. Further, we
highlight a range of potential future research directions. One of the unique
features of this review is that all software implementations of these GAN
methods and datasets have been collected and made available in one place at
https://github.com/pshams55/GAN-Case-Study.

negative:
We present an algorithm to directly solve numerous image restoration problems
(e.g., image deblurring, image dehazing, image deraining, etc.). These problems
are highly ill-posed, and the common assumptions for existing methods are
usually based on heuristic image priors. In this paper, we find that these
problems can be solved by generative models with adversarial learning. However,
the basic formulation of generative adversarial networks (GANs) does not
generate realistic images, and some structures of the estimated images are
usually not preserved well. Motivated by an interesting observation that the
estimated results should be consistent with the observed inputs under the
physics models, we propose a physics model constrained learning algorithm so
that it can guide the estimation of the specific task in the conventional GAN
framework. The proposed algorithm is trained in an end-to-end fashion and can
be applied to a variety of image restoration and related low-level vision
problems. Extensive experiments demonstrate that our method performs favorably
against the state-of-the-art algorithms.

negative:
As we develop more assistive and automated game design systems, the question
of how these systems should be integrated into game development workflows, and
how much adaptation may be required, becomes increasingly important. In this
paper we explore the impact of software engineering decisions on the ability of
an automated game design system to understand a game's codebase, generate new
game code, and evaluate its work. We argue that a new approach to software
engineering may be required in order for game developers to fully benefit from
automated game designers.

negative:
A deep neural network (DNN)-based speech enhancement (SE) aiming to maximize
the performance of an automatic speech recognition (ASR) system is proposed in
this paper. In order to optimize the DNN-based SE model in terms of the
character error rate (CER), which is one of the metric to evaluate the ASR
system and generally non-differentiable, our method uses two DNNs: one for
speech processing and one for mimicking the output CERs derived through an
acoustic model (AM). Then both of DNNs are alternately optimized in the
training phase. Even if the AM is a black-box, e.g., like one provided by a
third-party, the proposed method enables the DNN-based SE model to be optimized
in terms of the CER since the DNN mimicking the AM is differentiable.
Consequently, it becomes feasible to build CER-centric SE model that has no
negative effect, e.g., additional calculation cost and changing network
architecture, on the inference phase since our method is merely a training
scheme for the existing DNN-based methods. Experimental results show that our
method improved CER by 8.8% relative derived through a black-box AM although
certain noise levels are kept.

negative:
Our paper explores the game theoretic value of the 7-in-a-row game. We reduce
the problem to solving a finite board game, which we target using Proof Number
Search. We present a number of heuristic improvements to Proof Number Search
and examine their effect within the context of this particular game. Although
our paper does not solve the 7-in-a-row game, our experiments indicate that we
have made significant progress towards it.

negative:
This paper describes a new implementation of Planet Wars, designed from the
outset for Game AI research. The skill-depth of the game makes it a challenge
for game-playing agents, and the speed of more than 1 million game ticks per
second enables rapid experimentation and prototyping. The parameterised nature
of the game together with an interchangeable actuator model make it well suited
to automated game tuning. The game is designed to be fun to play for humans,
and is directly playable by General Video Game AI agents.

negative:
Generative adversarial networks (GANs) has gained tremendous popularity
lately due to an ability to reinforce quality of its predictive model with
generated objects and the quality of the generative model with and supervised
feedback. GANs allow to synthesize images with a high degree of realism.
However, the learning process of such models is a very complicated optimization
problem and certain limitation for such models were found. It affects the
choice of certain layers and nonlinearities when designing architectures. In
particular, it does not allow to train convolutional GAN models with
fully-connected hidden layers. In our work, we propose a modification of the
previously described set of rules, as well as new approaches to designing
architectures that will allow us to train more powerful GAN models. We show the
effectiveness of our methods on the problem of synthesizing projections of 3D
objects with the possibility of interpolation by class and view point.

negative:
One of the most significant challenges in statistical signal processing and
machine learning is how to obtain a generative model that can produce samples
of large-scale data distribution, such as images and speeches. Generative
Adversarial Network (GAN) is an effective method to address this problem. The
GANs provide an appropriate way to learn deep representations without
widespread use of labeled training data. This approach has attracted the
attention of many researchers in computer vision since it can generate a large
amount of data without precise modeling of the probability density function
(PDF). In GANs, the generative model is estimated via a competitive process
where the generator and discriminator networks are trained simultaneously. The
generator learns to generate plausible data, and the discriminator learns to
distinguish fake data created by the generator from real data samples. Given
the rapid growth of GANs over the last few years and their application in
various fields, it is necessary to investigate these networks accurately. In
this paper, after introducing the main concepts and the theory of GAN, two new
deep generative models are compared, the evaluation metrics utilized in the
literature and challenges of GANs are also explained. Moreover, the most
remarkable GAN architectures are categorized and discussed. Finally, the
essential applications in computer vision are examined.

negative:
Differentiable rendering has paved the way to training neural networks to
perform "inverse graphics" tasks such as predicting 3D geometry from monocular
photographs. To train high performing models, most of the current approaches
rely on multi-view imagery which are not readily available in practice. Recent
Generative Adversarial Networks (GANs) that synthesize images, in contrast,
seem to acquire 3D knowledge implicitly during training: object viewpoints can
be manipulated by simply manipulating the latent codes. However, these latent
codes often lack further physical interpretation and thus GANs cannot easily be
inverted to perform explicit 3D reasoning. In this paper, we aim to extract and
disentangle 3D knowledge learned by generative models by utilizing
differentiable renderers. Key to our approach is to exploit GANs as a
multi-view data generator to train an inverse graphics network using an
off-the-shelf differentiable renderer, and the trained inverse graphics network
as a teacher to disentangle the GAN's latent code into interpretable 3D
properties. The entire architecture is trained iteratively using cycle
consistency losses. We show that our approach significantly outperforms
state-of-the-art inverse graphics networks trained on existing datasets, both
quantitatively and via user studies. We further showcase the disentangled GAN
as a controllable 3D "neural renderer", complementing traditional graphics
renderers.

negative:
Generative Adversarial Networks (GANs) are commonly used for modeling complex
distributions of data. Both the generators and discriminators of GANs are often
modeled by neural networks, posing a non-transparent optimization problem which
is non-convex and non-concave over the generator and discriminator,
respectively. Such networks are often heuristically optimized with gradient
descent-ascent (GDA), but it is unclear whether the optimization problem
contains any saddle points, or whether heuristic methods can find them in
practice. In this work, we analyze the training of Wasserstein GANs with
two-layer neural network discriminators through the lens of convex duality, and
for a variety of generators expose the conditions under which Wasserstein GANs
can be solved exactly with convex optimization approaches, or can be
represented as convex-concave games. Using this convex duality interpretation,
we further demonstrate the impact of different activation functions of the
discriminator. Our observations are verified with numerical results
demonstrating the power of the convex interpretation, with applications in
progressive training of convex architectures corresponding to linear generators
and quadratic-activation discriminators for CelebA image generation. The code
for our experiments is available at https://github.com/ardasahiner/ProCoGAN.

negative:
The Generative Models have gained considerable attention in the field of
unsupervised learning via a new and practical framework called Generative
Adversarial Networks (GAN) due to its outstanding data generation capability.
Many models of GAN have proposed, and several practical applications emerged in
various domains of computer vision and machine learning. Despite GAN's
excellent success, there are still obstacles to stable training. The problems
are due to Nash-equilibrium, internal covariate shift, mode collapse, vanishing
gradient, and lack of proper evaluation metrics. Therefore, stable training is
a crucial issue in different applications for the success of GAN. Herein, we
survey several training solutions proposed by different researchers to
stabilize GAN training. We survey, (I) the original GAN model and its modified
classical versions, (II) detail analysis of various GAN applications in
different domains, (III) detail study about the various GAN training obstacles
as well as training solutions. Finally, we discuss several new issues as well
as research outlines to the topic.

negative:
Rehabilitation robotics combined with video game technology provides a means
of assisting in the rehabilitation of patients with neuromuscular disorders by
performing various facilitation movements. The current work presents ReHabGame,
a serious game using a fusion of implemented technologies that can be easily
used by patients and therapists to assess and enhance sensorimotor performance
and also increase the activities in the daily lives of patients. The game
allows a player to control avatar movements through a Kinect Xbox, Myo armband
and rudder foot pedal, and involves a series of reach-grasp-collect tasks whose
difficulty levels are learnt by a fuzzy interface. The orientation, angular
velocity, head and spine tilts and other data generated by the player are
monitored and saved, whilst the task completion is calculated by solving an
inverse kinematics algorithm which orientates the upper limb joints of the
avatar. The different values in upper body quantities of movement provide fuzzy
input from which crisp output is determined and used to generate an appropriate
subsequent rehabilitation game level. The system can thus provide personalised,
autonomously-learnt rehabilitation programmes for patients with neuromuscular
disorders with superior predictions to guide the development of improved
clinical protocols compared to traditional theraputic activities.

negative:
In this paper we introduce DeepCrawl, a fully-playable Roguelike prototype
for iOS and Android in which all agents are controlled by policy networks
trained using Deep Reinforcement Learning (DRL). Our aim is to understand
whether recent advances in DRL can be used to develop convincing behavioral
models for non-player characters in videogames. We begin with an analysis of
requirements that such an AI system should satisfy in order to be practically
applicable in video game development, and identify the elements of the DRL
model used in the DeepCrawl prototype. The successes and limitations of
DeepCrawl are documented through a series of playability tests performed on the
final game. We believe that the techniques we propose offer insight into
innovative new avenues for the development of behaviors for non-player
characters in video games, as they offer the potential to overcome critical
issues with

negative:
Many games often share common ideas or aspects between them, such as their
rules, controls, or playing area. However, in the context of General Game
Playing (GGP) for board games, this area remains under-explored. We propose to
formalise the notion of "game concept", inspired by terms generally used by
game players and designers. Through the Ludii General Game System, we describe
concepts for several levels of abstraction, such as the game itself, the moves
played, or the states reached. This new GGP feature associated with the ludeme
representation of games opens many new lines of research. The creation of a
hyper-agent selector, the transfer of AI learning between games, or explaining
AI techniques using game terms, can all be facilitated by the use of game
concepts. Other applications which can benefit from game concepts are also
discussed, such as the generation of plausible reconstructed rules for
incomplete ancient games, or the implementation of a board game recommender
system.

negative:
Coreference evaluation metrics are hard to optimize directly as they are
non-differentiable functions, not easily decomposable into elementary
decisions. Consequently, most approaches optimize objectives only indirectly
related to the end goal, resulting in suboptimal performance. Instead, we
propose a differentiable relaxation that lends itself to gradient-based
optimisation, thus bypassing the need for reinforcement learning or heuristic
modification of cross-entropy. We show that by modifying the training objective
of a competitive neural coreference system, we obtain a substantial gain in
performance. This suggests that our approach can be regarded as a viable
alternative to using reinforcement learning or more computationally expensive
imitation learning.

negative:
If you are an artificial intelligence researcher, you should look to video
games as ideal testbeds for the work you do. If you are a video game developer,
you should look to AI for the technology that makes completely new types of
games possible. This chapter lays out the case for both of these propositions.
It asks the question "what can video games do for AI", and discusses how in
particular general video game playing is the ideal testbed for artificial
general intelligence research. It then asks the question "what can AI do for
video games", and lays out a vision for what video games might look like if we
had significantly more advanced AI at our disposal. The chapter is based on my
keynote at IJCCI 2015, and is written in an attempt to be accessible to a broad
audience.

negative:
This project proposes a methodology for the automatic generation of action
models from video game dynamics descriptions, as well as its integration with a
planning agent for the execution and monitoring of the plans. Planners use
these action models to get the deliberative behaviour for an agent in many
different video games and, combined with a reactive module, solve deterministic
and no-deterministic levels. Experimental results validate the methodology and
prove that the effort put by a knowledge engineer can be greatly reduced in the
definition of such complex domains. Furthermore, benchmarks of the domains has
been produced that can be of interest to the international planning community
to evaluate planners in international planning competitions.

negative:
Over the past few years the Angry Birds AI competition has been held in an
attempt to develop intelligent agents that can successfully and efficiently
solve levels for the video game Angry Birds. Many different agents and
strategies have been developed to solve the complex and challenging physical
reasoning problems associated with such a game. However none of these agents
attempt one of the key strategies which humans employ to solve Angry Birds
levels, which is restarting levels. Restarting is important in Angry Birds
because sometimes the level is no longer solvable or some given shot made has
little to no benefit towards the ultimate goal of the game. This paper proposes
a framework and experimental evaluation for when to restart levels in Angry
Birds. We demonstrate that restarting is a viable strategy to improve agent
performance in many cases.

negative:
We propose the study of mathematical ludology, which aims to formally
interrogate questions of interest to game studies and game design in
particular. The goal is to extend our mathematical understanding of complex
games beyond decision-making---the typical focus of game theory and artificial
intelligence efforts---to explore other aspects such as game mechanics,
structure, relationships between games, and connections between game rules and
user-interfaces, as well as exploring related gameplay phenomena and typical
player behavior. In this paper, we build a basic foundation for this line of
study by developing a hierarchy of game descriptions, mathematical formalism to
compactly describe complex discrete games, and equivalence relations on the
space of game systems.

negative:
Game-playing AI research has focused for a long time on learning to play
video games from visual input or symbolic information. However, humans benefit
from a wider array of sensors which we utilise in order to navigate the world
around us. In particular, sounds and music are key to how many of us perceive
the world and influence the decisions we make. In this paper, we present
initial experiments on game-playing agents learning to play video games solely
from audio cues. We expand the Video Game Description Language to allow for
audio specification, and the General Video Game AI framework to provide new
audio games and an API for learning agents to make use of audio observations.
We analyse the games and the audio game design process, include initial results
with simple Q~Learning agents, and encourage further research in this area.

negative:
Class labels have been empirically shown useful in improving the sample
quality of generative adversarial nets (GANs). In this paper, we mathematically
study the properties of the current variants of GANs that make use of class
label information. With class aware gradient and cross-entropy decomposition,
we reveal how class labels and associated losses influence GAN's training.
Based on that, we propose Activation Maximization Generative Adversarial
Networks (AM-GAN) as an advanced solution. Comprehensive experiments have been
conducted to validate our analysis and evaluate the effectiveness of our
solution, where AM-GAN outperforms other strong baselines and achieves
state-of-the-art Inception Score (8.91) on CIFAR-10. In addition, we
demonstrate that, with the Inception ImageNet classifier, Inception Score
mainly tracks the diversity of the generator, and there is, however, no
reliable evidence that it can reflect the true sample quality. We thus propose
a new metric, called AM Score, to provide a more accurate estimation of the
sample quality. Our proposed model also outperforms the baseline methods in the
new metric.

negative:
Generative Adversarial Networks (GANs) is a novel class of deep generative
models which has recently gained significant attention. GANs learns complex and
high-dimensional distributions implicitly over images, audio, and data.
However, there exists major challenges in training of GANs, i.e., mode
collapse, non-convergence and instability, due to inappropriate design of
network architecture, use of objective function and selection of optimization
algorithm. Recently, to address these challenges, several solutions for better
design and optimization of GANs have been investigated based on techniques of
re-engineered network architectures, new objective functions and alternative
optimization algorithms. To the best of our knowledge, there is no existing
survey that has particularly focused on broad and systematic developments of
these solutions. In this study, we perform a comprehensive survey of the
advancements in GANs design and optimization solutions proposed to handle GANs
challenges. We first identify key research issues within each design and
optimization technique and then propose a new taxonomy to structure solutions
by key research issues. In accordance with the taxonomy, we provide a detailed
discussion on different GANs variants proposed within each solution and their
relationships. Finally, based on the insights gained, we present the promising
research directions in this rapidly growing field.

negative:
Generative adversarial network (GAN) is among the most popular deep learning
models for learning complex data distributions. However, training a GAN is
known to be a challenging task. This is often attributed to the lack of
correlation between the training progress and the trajectory of the generator
and discriminator losses and the need for the GAN's subjective evaluation. A
recently proposed measure inspired by game theory - the duality gap, aims to
bridge this gap. However, as we demonstrate, the duality gap's capability
remains constrained due to limitations posed by its estimation process. This
paper presents a theoretical understanding of this limitation and proposes a
more dependable estimation process for the duality gap. At the crux of our
approach is the idea that local perturbations can help agents in a zero-sum
game escape non-Nash saddle points efficiently. Through exhaustive
experimentation across GAN models and datasets, we establish the efficacy of
our approach in capturing the GAN training progress with minimal increase to
the computational complexity. Further, we show that our estimate, with its
ability to identify model convergence/divergence, is a potential performance
measure that can be used to tune the hyperparameters of a GAN.

negative:
Generative Adversarial Networks (GANs) have become a dominant class of
generative models. In recent years, GAN variants have yielded especially
impressive results in the synthesis of a variety of forms of data. Examples
include compelling natural and artistic images, textures, musical sequences,
and 3D object files. However, one obvious synthesis candidate is missing. In
this work, we answer one of deep learning's most pressing questions: GAN you do
the GAN GAN? That is, is it possible to train a GAN to model a distribution of
GANs? We release the full source code for this project under the MIT license.

negative:
Generative Adversarial Networks (GAN) have received wide attention in the
machine learning field for their potential to learn high-dimensional, complex
real data distribution. Specifically, they do not rely on any assumptions about
the distribution and can generate real-like samples from latent space in a
simple manner. This powerful property leads GAN to be applied to various
applications such as image synthesis, image attribute editing, image
translation, domain adaptation and other academic fields. In this paper, we aim
to discuss the details of GAN for those readers who are familiar with, but do
not comprehend GAN deeply or who wish to view GAN from various perspectives. In
addition, we explain how GAN operates and the fundamental meaning of various
objective functions that have been suggested recently. We then focus on how the
GAN can be combined with an autoencoder framework. Finally, we enumerate the
GAN variants that are applied to various tasks and other fields for those who
are interested in exploiting GAN for their research.

negative:
This paper describes a new algorithm for decision making in two-player
real-time video games. As with Monte Carlo Tree Search, the algorithm can be
used without heuristics and has been developed for use in general video game
AI. The approach is to extend recent work on rolling horizon evolutionary
planning, which has been shown to work well for single-player games, to two (or
in principle many) player games. To select an action the algorithm co-evolves
two (or in the general case N) populations, one for each player, where each
individual is a sequence of actions for the respective player. The fitness of
each individual is evaluated by playing it against a selection of
action-sequences from the opposing population. When choosing an action to take
in the game, the first action is chosen from the fittest member of the
population for that player. The new algorithm is compared with a number of
general video game AI algorithms on three variations of a two-player space
battle game, with promising results.

negative:
An approach for game bot detection in MMORPGs is proposed based on the
analysis of game playing behavior. Since MMORPGs are large scale games, users
can play in various ways. This variety in playing behavior makes it hard to
detect game bots based on play behaviors. In order to cope with this problem,
the proposed approach observes game playing behaviors of users and groups them
by their behavioral similarities. Then, it develops a local bot detection model
for each player group. Since the locally optimized models can more accurately
detect game bots within each player group, the combination of those models
brings about overall improvement. For a practical purpose of reducing the
workloads of the game servers in service, the game data is collected at a low
resolution in time. Behavioral features are selected and developed to
accurately detect game bots with the low resolution data, considering common
aspects of MMORPG playing. Through the experiment with the real data from a
game currently in service, it is shown that the proposed local model approach
yields more accurate results.

negative:
The game industry has long been troubled by malicious activities utilizing
game bots. The game bots disturb other game players and destroy the
environmental system of the games. For these reasons, the game industry put
their best efforts to detect the game bots among players' characters using the
learning-based detections. However, one problem with the detection
methodologies is that they do not provide rational explanations about their
decisions. To resolve this problem, in this work, we investigate the
explainabilities of the game bot detection. We develop the XAI model using a
dataset from the Korean MMORPG, AION, which includes game logs of human players
and game bots. More than one classification model has been applied to the
dataset to be analyzed by applying interpretable models. This provides us
explanations about the game bots' behavior, and the truthfulness of the
explanations has been evaluated. Besides, interpretability contributes to
minimizing false detection, which imposes unfair restrictions on human players.

negative:
This paper presents TotalBotWar, a new pseudo real-time multi-action
challenge for game AI, as well as some initial experiments that benchmark the
framework with different agents. The game is based on the real-time battles of
the popular TotalWar games series where players manage an army to defeat the
opponent's one. In the proposed game, a turn consists of a set of orders to
control the units. The number and specific orders that can be performed in a
turn vary during the progression of the game. One interesting feature of the
game is that if a particular unit does not receive an order in a turn, it will
continue performing the action specified in a previous turn. The turn-wise
branching factor becomes overwhelming for traditional algorithms and the
partial observability of the game state makes the proposed game an interesting
platform to test modern AI algorithms.

negative:
Modern board games are a rich source of interesting and new challenges for
combinatorial problems. The game Nmbr9 is a solitaire style puzzle game using
polyominoes. The rules of the game are simple to explain, but modelling the
game effectively using constraint programming is hard. This abstract presents
the game, contributes new generalized variants of the game suitable for
benchmarking and testing, and describes a model for the presented variants. The
question of the top possible score in the standard game is an open challenge.

negative:
Cross-modal retrieval between videos and texts has attracted growing
attentions due to the rapid emergence of videos on the web. The current
dominant approach for this problem is to learn a joint embedding space to
measure cross-modal similarities. However, simple joint embeddings are
insufficient to represent complicated visual and textual details, such as
scenes, objects, actions and their compositions. To improve fine-grained
video-text retrieval, we propose a Hierarchical Graph Reasoning (HGR) model,
which decomposes video-text matching into global-to-local levels. To be
specific, the model disentangles texts into hierarchical semantic graph
including three levels of events, actions, entities and relationships across
levels. Attention-based graph reasoning is utilized to generate hierarchical
textual embeddings, which can guide the learning of diverse and hierarchical
video representations. The HGR model aggregates matchings from different
video-text levels to capture both global and local details. Experimental
results on three video-text datasets demonstrate the advantages of our model.
Such hierarchical decomposition also enables better generalization across
datasets and improves the ability to distinguish fine-grained semantic
differences.