@JuanFMontesinos 's answer will work, but only in case your use case is model (CPU or single GPU) -> model (multiple GPUs) (multi GPU to multi GPU should work normally, as should CPU / single GPU to CPU / single GPU).
If the use case is model (multiple GPUs) -> model(CPU or single GPU), you need this hack if the model was already saved:
substring = 'module.'
checkpoint_tmp = OrderedDict()
for k in checkpoint:
    new_k = k[len(substring):] if k.startswith(substring) else k
    checkpoint_tmp[new_k] = checkpoint[k]
checkpoint = checkpoint_tmp

This will remove the module. prefix that is appended to DataParallel models and load normally afterwards.
If the model was not already saved, you can prevent this from happening in the future by saving as follows (taken from this issue 12):
try:
    model_state_dict = model.module.state_dict()
except AttributeError:
    model_state_dict = model.state_dict()

which will remove the module. prefix before saving normally.