Definition: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
Input: Low dimensional representations of words allow<lb>accurate NLP models to be trained on limited<lb>annotated data. While most representations ig-<lb>nore words’ local context, a natural way to in-<lb>duce context-dependent representations is to per-<lb>form inference in a probabilistic latent-variable<lb>sequence model. Given the recent success of<lb>continuous vector space word representations,<lb>we provide such an inference procedure for con-<lb>tinuous states, where words’ representations are<lb>given by the posterior mean of a linear dynam-<lb>ical system. Here, efficient inference can be<lb>performed using Kalman filtering. Our learn-<lb>ing algorithm is extremely scalable, operating<lb>on simple cooccurrence counts for both param-<lb>eter initialization using the method of moments<lb>and subsequent iterations of EM. In our exper-<lb>iments, we employ our inferred word embed-<lb>dings as features in standard tagging tasks, ob-<lb>taining significant accuracy improvements. Fi-<lb>nally, the Kalman filter updates can be seen as a<lb>linear recurrent neural network. We demonstrate<lb>that using the parameters of our model to ini-<lb>tialize a non-linear recurrent neural network lan-<lb>guage model reduces its training time by a day<lb>and yields lower perplexity.

A Linear Dynamical System Model for Text