Definition: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
Input: We introduce the “exponential linear unit” (ELU) which speeds up learning in<lb>deep neural networks and leads to higher classification accuracies. Like rectified<lb>linear units (ReLUs), leaky ReLUs (LReLUs) and parametrized ReLUs (PRe-<lb>LUs), ELUs also avoid a vanishing gradient via the identity for positive values.<lb>However ELUs have improved learning characteristics compared to the units with<lb>other activation functions. In contrast to ReLUs, ELUs have negative values which<lb>allows them to push mean unit activations closer to zero. Zero means speed up<lb>learning because they bring the gradient closer to the unit natural gradient. We<lb>show that the unit natural gradient differs from the normal gradient by a bias shift<lb>term, which is proportional to the mean activation of incoming units. Like batch<lb>normalization, ELUs push the mean towards zero, but with a significantly smaller<lb>computational footprint. While other activation functions like LReLUs and PRe-<lb>LUs also have negative values, they do not ensure a noise-robust deactivation state.<lb>ELUs saturate to a negative value with smaller inputs and thereby decrease the<lb>propagated variation and information. Therefore ELUs code the degree of pres-<lb>ence of particular phenomena in the input, while they do not quantitatively model<lb>the degree of their absence. Consequently, dependencies between ELUs are much<lb>easier to model and distinct concepts are less likely to interfere.<lb>We found that ELUs lead not only to faster learning, but also to better general-<lb>ization performance once networks have many layers (≥ 5). ELU networks were<lb>among top 10 reported CIFAR-10 results and yielded the best published result on<lb>CIFAR-100, without resorting to multi-view evaluation or model averaging. On<lb>ImageNet, ELU networks considerably speed up learning compared to a ReLU<lb>network with the same architecture, obtaining less than 10% classification error<lb>for a single crop, single model network.

EXPONENTIAL LINEAR UNITS (ELUS)