I think your first fully connected layer is a bit big no? the weight size is `torch.Size([1000, 2408448]). Meaning that the input feature size is more than 2 million!
I think that you should reduce the size of that layer and it will help drastically with memory usage.
You can add extra pooling or striding in the last convs to reduce this size.