Reframing a HMM problem as an RNN Inspired by this question I have been considering how one would reframe a HMM problem as RNN problem.
For HMMs we have some observable timeseries $y(t)$ which corresponds to a set of hidden states $q(t)$. The transition probabilities between hidden states are described by a probability matrix $\bar{A}$. For a Markov process, it is possible to derive an expression for the probability that the hidden path $Q$ corresponds to the observation $O$ i.e. $P(Q|O)$. One can then use some optimisation algorithm -e.g. Viterbi to find the most probable path $Q*$ which maximises $P(Q|O)$.
But how does this formulation translate to an RNN?
With RNNs we talk in the language of training data, nodes, layers etc. and I can't see how to map this to the HMM language...

Neural networks can be used to amortize the optimization part, effectively learning an adaptive solution given a corpus of data.
The connection with VAEs is pretty easy to see here.
So, in your notation, instead of optimizing for $Q*$, you would learn an approximate posterior.
See the Structured Inference Networks and Deep Markov Models presented in:

R. G. Krishnan, U. Shalit, and D. Sontag, “Structured Inference Networks for Nonlinear State Space Models.” 31st AAAI Conference on Artificial Intelligence, AAAI, 2017. doi: 10.48550/ARXIV.1609.09869.

