Thanks, I changed it to
        thre=thre.to('cuda:0')
        mask = weight_copy.abs().gt(thre).float()

but started getting out of memory errors!, Then I used   with torch.no_grad(): and that was solved. however I now get this error :
Pre-processing Successful!
Files already downloaded and verified
prune.py:112: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  data, target = Variable(data, volatile=True), Variable(target)

Test set: Accuracy: 1000/10000 (10.0%)

cfg/simpnet
idx0 values : [0 1 2] idx1 values :[ 0  1  3  5  8  9 18 20 21 26 27 31 33 34 41 46 50 51 54 57 59 65]
idx0 shape: (3,) idx1 shape:(22,)
In shape: 3 Out shape:22
type:  <class 'numpy.ndarray'>
Traceback (most recent call last):
  File "prune.py", line 136, in <module>
    m1.weight.data = m0.weight.data[idx1].clone()
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in __getattr__
    type(self).__name__, name))
AttributeError: 'ReLU' object has no attribute 'weight'