I wrote a fresh code to check whether the models are being updated or not. I First use_amp=True and then run the script and again (I am simply doing python3 main.py from the terminal). I did this for 10 times and found that only 3 times, I get (False, False …).
When I do not use amp I always get valid updates. This is the behaviour on my machine(Though I dont think it should matter). I even deleted the cached files just in case.
If you keep rerunning(not a loop, a fresh run of the script every time) that piece code, Even you might get True True True True at least once. Are you seeing such behaviour on your machine?