  We explore in depth how categorical data can be processed with embeddings in
the context of claim severity modeling. We develop several models that range in
complexity from simple neural networks to state-of-the-art attention based
architectures that utilize embeddings. We illustrate the utility of learned
embeddings from neural networks as pretrained features in generalized linear
models, and discuss methods for visualizing and interpreting embeddings.
Finally, we explore how attention based models can contextually augment
embeddings, leading to enhanced predictive performance.

Applications;Risk Management