Hi everyone, pytorch and MLnewbie here,
I’m trying to do matrix factorization via SGD, where each row in my dataset describes one entry in the target matrix. I’m trying to make use of my GPU to maximize parallel computation. In order to achieve this, I know that a larger batch must be used. I’m running into an issue where I need to consider a batch size and accuracy tradeoff. The larger the batch size, the more the GPU can be utilized, but in exchange my model converges to a local minima due to a lack of noise in the batch.
So my question is, is there some way where I can achieve the same performance gains of using a larger batch sizes while retaining the convergence  results of smaller batch sizes?
Any input would be appreciated. Thanks!