Dear all,
I’m trying to implement the Neural-expectation maximisation architecture in pytorch.
Unfortunately, such a model requires a VanillaRNN without any activation function (or a sigmoid activation function). Currently, the pytorch._backend library seems to support only RNNs with tanh or ReLU activations. However, the RNNBase module is not documented but appears to support different execution modes. I’m not sure if such modes are just LSTM, GRU, RNN_tanh and RNN_ReLU.
Does anyone know if calling the RNNBase module with an empty mode works and in case what should I expect as output?
If I have to implement a simple RNN without activation in python-native, does anyone know how slow would it be w.r.t. the C implementation?
Cheers,
Sandro