text: Title: Training on older graphics card with custom dataset.
  Body: Hey,
  I am using the Common-Voice dataset for German language and I am trying to train a synthesizer on my old GTX 660TI with 2G VRAM (Unfortunately :D). 
  As Cuda Compute Capability 3.0 is not supported by Pytorch, I always built it from source. I've tried different versions of pytorch(1.4, 1.5, 1.7 and 1.10) and of cuda (10.1 and 10.2.) and cudnn(8.1 and 7.5). So finally I am able to train, even with a very low batch size of 6. 
  I have a few questions:
  
  1: Do you think I can achieve decent results with this low batch size? I've trained to around 100k steps with a loss now of around 0.63, and it feels like it is converging now.
  
  2: How many utterances should I use from every speaker. In my dataset, there are some speakers with thousands of utterances, is it better to shrink the number? For example, is it better using e.g. 200 utterances of each speaker?
   
  3: When training, after a few steps (completely random, sometimes after 10, sometimes after 1000) I get a NaN loss and the error:
  
  `.../cuda/Loss.cu:111: block: [7,0,0], thread: [33,0,0] Assertion `input_val >= zero && input_val <= one` failed. `
  
  Do you think it is caused by my old graphics card? I've read that is maybe caused by non-normalized data, but I am not sure what I should normalize here. I didn't find any issues regarding that here.
  
  Thanks in advance :)