Hi ngimel,
Thank you so much for your reply!
I do comment out the if input.is_cuda section, and when I run the network normally, my modifications are having an effect.
but when I activate Cuda (e.g. USE_GPU = True), the code runs super fast, but it seems to entirely skip the entire function (I can still comment out that section and it still runs with Cuda).
I have even put the ‘print(‘working’)’ line inside the if input.is_cuda loop (when cuda is activated) and it still does not print.
To clarify, my extension works perfectly fine if I do not activate cuda. But it seems that turning my model and tensors into cuda form starts a process in which the LSTM cell architecture is generated by cuda.
I have traced the information flow to this class: class CudnnRNN(NestedIOFunction), within the same nn_functions\rnn.py file, and from there the information flows to torch\backends\cudnn\rnn.py, but nowhere do I find Cuda building the actual architecture for me to modify.
But I am very new to coding, so perhaps I misunderstood what you meant by creating my own extension.