A single sample of [ 1024, 1024]  input takes up almost my entire GTX1080TI GPU, that’s roughly 10G+ memory.
Also, I wonder if there is any implementation error in my code? Cause I’ve seen a bigger model that can handle  [ 1024, 1024]  input with less memory.