Adam is not suited for sparse gradients, because Adam has a step that densifies things.
Try using RMSProp instead.
This thread has more context:

  
      github.com/pytorch/pytorch
  
  
    



Issue: sparse optimizers 195


	
	opened by adamlerer
	on 2017-04-18
	
	
	


Now that sparse tensors are mostly working (#1147), it would be awesome if all the optimizers worked with sparse tensors. This...


 	enhancement
 	medium priority