Definition: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
Input: Reactive (memoryless) policies are sufficient in completely observable Markov decision pro­ cesses (MDPs), but some kind of memory is usually necessary for optimal control of a par­ tially observable MDP. Policies with finite mem­ ory can be represented as finite-state automata. In this paper, we extend Baird and Moore's YAPS algorithm to the problem of learning gen­ eral finite-state automata. Because it performs stochastic gradient descent, this algorithm can be shown to converge to a locally optimal finite­ state controller. We provide the details of the algorithm and then consider the question of un­ der what conditions stochastic gradient descent will outperform exact gradient descent. We con­ clude with empirical results comparing the per­ formance of stochastic and exact gradient de­ scent, and showing the ability of our algorithm to extract the useful information contained in the sequence of past observations to compensate for the lack of observability at each time-step.

Learning Finite-State Controllers for Partially Observable Environments