It seems like everything is on cuda:0, but the error says something is on cuda:1 :
Printed from code above:
module.decoder.mlp_f0.mlp_layer1.0.weight cuda:0
module.decoder.mlp_f0.mlp_layer1.0.bias cuda:0
module.decoder.mlp_f0.mlp_layer1.1.weight cuda:0
module.decoder.mlp_f0.mlp_layer1.1.bias cuda:0
module.decoder.mlp_f0.mlp_layer2.0.weight cuda:0
module.decoder.mlp_f0.mlp_layer2.0.bias cuda:0
module.decoder.mlp_f0.mlp_layer2.1.weight cuda:0
module.decoder.mlp_f0.mlp_layer2.1.bias cuda:0
module.decoder.mlp_f0.mlp_layer3.0.weight cuda:0
module.decoder.mlp_f0.mlp_layer3.0.bias cuda:0
module.decoder.mlp_f0.mlp_layer3.1.weight cuda:0
module.decoder.mlp_f0.mlp_layer3.1.bias cuda:0
module.decoder.mlp_loudness.mlp_layer1.0.weight cuda:0
module.decoder.mlp_loudness.mlp_layer1.0.bias cuda:0
module.decoder.mlp_loudness.mlp_layer1.1.weight cuda:0
module.decoder.mlp_loudness.mlp_layer1.1.bias cuda:0
module.decoder.mlp_loudness.mlp_layer2.0.weight cuda:0
module.decoder.mlp_loudness.mlp_layer2.0.bias cuda:0
module.decoder.mlp_loudness.mlp_layer2.1.weight cuda:0
module.decoder.mlp_loudness.mlp_layer2.1.bias cuda:0
module.decoder.mlp_loudness.mlp_layer3.0.weight cuda:0
module.decoder.mlp_loudness.mlp_layer3.0.bias cuda:0
module.decoder.mlp_loudness.mlp_layer3.1.weight cuda:0
module.decoder.mlp_loudness.mlp_layer3.1.bias cuda:0
module.decoder.gru.weight_ih_l0 cuda:0
module.decoder.gru.weight_hh_l0 cuda:0
module.decoder.gru.bias_ih_l0 cuda:0
module.decoder.gru.bias_hh_l0 cuda:0
module.decoder.mlp_gru.mlp_layer1.0.weight cuda:0
module.decoder.mlp_gru.mlp_layer1.0.bias cuda:0
module.decoder.mlp_gru.mlp_layer1.1.weight cuda:0
module.decoder.mlp_gru.mlp_layer1.1.bias cuda:0
module.decoder.mlp_gru.mlp_layer2.0.weight cuda:0
module.decoder.mlp_gru.mlp_layer2.0.bias cuda:0
module.decoder.mlp_gru.mlp_layer2.1.weight cuda:0
module.decoder.mlp_gru.mlp_layer2.1.bias cuda:0
module.decoder.mlp_gru.mlp_layer3.0.weight cuda:0
module.decoder.mlp_gru.mlp_layer3.0.bias cuda:0
module.decoder.mlp_gru.mlp_layer3.1.weight cuda:0
module.decoder.mlp_gru.mlp_layer3.1.bias cuda:0
module.decoder.dense_harmonic.weight cuda:0
module.decoder.dense_harmonic.bias cuda:0
module.decoder.dense_filter.weight cuda:0
module.decoder.dense_filter.bias cuda:0
module.reverb.fir cuda:0
module.reverb.drywet cuda:0
module.reverb.decay cuda:0

Error:
 File "/ddsp-pytorch/train/trainer/trainer.py", line 246, in train
    phase, num_steps=num_steps
  File "/ddsp-pytorch/train/trainer/trainer.py", line 377, in _train
    results = self._step(phase=phase, iterator=step_iterator)
  File "/ddsp-pytorch/train/trainer/trainer.py", line 336, in _step
    outputs = self.net(batch_dict)
  File ".local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/.local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 161, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/.local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 171, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
  File "/.local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
    output.reraise()
  File "/.local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise
    raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in replica 1 on device 1.
Original Traceback (most recent call last):
  File "/.local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/.local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/ddsp-pytorch/train/network/autoencoder/autoencoder.py", line 64, in forward
    batch = self.encoder(batch)
  File /.local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "ddsp-pytorch/train/network/autoencoder/encoder.py", line 101, in forward
    batch["loudness"] = self.loudness_extractor(batch)
  File "/.local/share/virtualenvs/ddsp-pytorch-BwyVRuzu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/ddsp-pytorch/train/../components/loudness_extractor.py", line 79, in forward
    sliced_windowed_signal = sliced_signal * self.smoothing_window
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!