determined or constrained by an underlying process logic, for example by business rules based on case data. Just as linguistic rules and constraints are not explicitly captured in NLP deep-learning approaches (Mikolov et al., 2011; Sutskever et al., 2011; Graves, 2012, 2013; Zaremba et al., 2014) but are learned by the neural net- work, the process constraints and rules also need not be explicitly represented but can be learned. 

However, while there are many similarities between natural language and process traces, there are important differences. First, the size of the vocabulary in process prediction is much smaller than the size of a natural language vocabulary. Second, the length of a trace can far exceed the typical sentence length in natural language. Together, these two differences result in fewer possible prediction targets (vocabulary size or number of unique process event types) and more information to predict from (sentence or trace prefix length), suggesting that this approach may be able to achieve better results than word prediction in NLP. Third, in contrast to natural language, processes contain activities with temporally overlapping execution. But even in that case, individual lifecycle events are strictly ordered, e.g. ”A—Start, B— Start, A—Complete, B—Complete”, as defined by the XES Working Group (2016): ”An event that occurs in a log . . . before another event that is related to the same trace, shall be assumed to have occurred before that other event.” However, the possibly arbitrary temporal sequence of events of parallel activities across different traces can make the prediction task more difficult, especially when there are no underlying regularities or dependencies imposed by other process characteristics such as resources or case attributes. 

To empirically examine the similarity of process event logs and natural language text, we focus on two characteristics. First, the mutual information I of two words in natural language declines with their distance d according to a power law ( I ∝ d − $^{k}$), 

