Hi Gabriel,
good catch! It does indeed not have anything to do with the embedding! It is a trick.
What it does it grabs any parameter of the model and uses it to instantiate (through .data.new) a new tensor on the same device (i.e. cpu if the model/its parameters are on cpu, the same gpu as the parameter if the model has been transferred with model.cuda()).
There are sometimes discussions how to do this (eg the issue below), but I must admit I’m not actually aware of a recommended way to write this efficiently.
Best regards
Thomas

  
      github.com/pytorch/pytorch
  
  
    



Issue: Device-agnostic code 96


	
	opened by Kaixhin
	on 2017-10-20
	
	
	


A common issue - for both new and more seasoned users, including myself - is how to write nice device-agnostic code....