Solved by ptrblck in post #2 
              
              
                You could probably use unfold to get the image patches and calculate the loss on these. 
Have a look at this post for an example.