Omer,
I just found your post, and given the subject matter it is none too long. In particular your followup with the amendment was very beneficial.
This will help a great deal; btw I reran the PyTorch script, both with Adam and BFGS optimizers and they work very well (quite fast even on my 2016 vintage Asus).
Going forward I will send more feedback as I go.
Thanks for sharing this material,
Jan