Ok first without the suggested changes
--------------------------------------------------------------------------------
  Environment Summary
--------------------------------------------------------------------------------
PyTorch 1.0.0 compiled w/ CUDA 10.0
Running with Python 3.6 and CUDA 9.0.176

`pip3 list` truncated output:
numpy (1.15.4)
torch (1.0.0)
torch-salad (0.2.1a0)
torchsummary (1.5.1)
torchvision (0.2.1)
--------------------------------------------------------------------------------
  cProfile output
--------------------------------------------------------------------------------
         14039572 function calls (13943858 primitive calls) in 196.499 seconds

   Ordered by: internal time
   List reduced from 7984 to 15 due to restriction <15>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      968  107.041    0.111  107.041    0.111 {method 'to' of 'torch._C._TensorBase' objects}
     1050   42.655    0.041   42.655    0.041 {method 'cpu' of 'torch._C._TensorBase' objects}
    39682    6.548    0.000    6.548    0.000 {method 'decode' of 'ImagingDecoder' objects}
      345    5.424    0.016    5.424    0.016 {method 'run_backward' of 'torch._C._EngineBase' objects}
    11550    2.932    0.000    2.932    0.000 {built-in method conv2d}
     3135    2.866    0.001    2.866    0.001 {method 'convert' of 'ImagingCore' objects}
      700    2.374    0.003    2.374    0.003 {built-in method stack}
    59340    2.366    0.000    2.366    0.000 {method 'mul_' of 'torch._C._TensorBase' objects}
    59340    2.250    0.000    2.250    0.000 {method 'add_' of 'torch._C._TensorBase' objects}
     2800    1.768    0.001    1.768    0.001 {method 'resize' of 'ImagingCore' objects}
    29670    1.683    0.000    1.683    0.000 {method 'sqrt' of 'torch._C._TensorBase' objects}
    35366    1.224    0.000    1.224    0.000 {built-in method nt.stat}
    29670    1.061    0.000    1.061    0.000 {method 'addcdiv_' of 'torch._C._TensorBase' objects}
     1551    1.033    0.001    2.266    0.001 C:\Users\Sklipnoty\AppData\Local\Programs\Python\Python36\lib\inspect.py:714(getmodule)
     7000    1.032    0.000    1.032    0.000 {built-in method batch_norm}


--------------------------------------------------------------------------------
  autograd profiler output (CPU mode)
--------------------------------------------------------------------------------
        top 15 events sorted by cpu_time_total

------  ---------------  ---------------  ---------------  ---------------  ---------------
Name           CPU time        CUDA time            Calls        CPU total       CUDA total
------  ---------------  ---------------  ---------------  ---------------  ---------------
to         380902.828us          0.000us                1     380902.828us          0.000us
to         376077.611us          0.000us                1     376077.611us          0.000us
to         374673.211us          0.000us                1     374673.211us          0.000us
to         372762.354us          0.000us                1     372762.354us          0.000us
to         372180.258us          0.000us                1     372180.258us          0.000us
to         371888.240us          0.000us                1     371888.240us          0.000us
to         371833.937us          0.000us                1     371833.937us          0.000us
to         371642.490us          0.000us                1     371642.490us          0.000us
to         370978.386us          0.000us                1     370978.386us          0.000us
to         370178.802us          0.000us                1     370178.802us          0.000us
to         370138.628us          0.000us                1     370138.628us          0.000us
to         369728.308us          0.000us                1     369728.308us          0.000us
to         369525.225us          0.000us                1     369525.225us          0.000us
to         367767.579us          0.000us                1     367767.579us          0.000us
to         367330.938us          0.000us                1     367330.938us          0.000us

--------------------------------------------------------------------------------
  autograd profiler output (CUDA mode)
--------------------------------------------------------------------------------
        top 15 events sorted by cpu_time_total

        Because the autograd profiler uses the CUDA event API,
        the CUDA time column reports approximately max(cuda_time, cpu_time).
        Please ignore this output if your code does not use CUDA.

------  ---------------  ---------------  ---------------  ---------------  ---------------
Name           CPU time        CUDA time            Calls        CPU total       CUDA total
------  ---------------  ---------------  ---------------  ---------------  ---------------
to         217067.124us       5554.688us                1     217067.124us       5554.688us
to         213935.831us       4132.812us                1     213935.831us       4132.812us
to         213064.211us       4640.625us                1     213064.211us       4640.625us
to         210899.292us       7593.750us                1     210899.292us       7593.750us
to         210715.881us       5867.188us                1     210715.881us       5867.188us
to         210535.517us       4320.312us                1     210535.517us       4320.312us
to         210234.910us       5375.000us                1     210234.910us       5375.000us
to         209686.062us       7722.656us                1     209686.062us       7722.656us
to         209682.460us       5093.750us                1     209682.460us       5093.750us
to         208417.697us       4187.500us                1     208417.697us       4187.500us
to         207810.943us       4703.125us                1     207810.943us       4703.125us
to         207540.813us       5859.375us                1     207540.813us       5859.375us
to         207383.722us       5031.250us                1     207383.722us       5031.250us
to         207044.882us       3968.750us                1     207044.882us       3968.750us
to         205328.794us       4980.469us                1     205328.794us       4980.469us


With the changes
--------------------------------------------------------------------------------
  Environment Summary
--------------------------------------------------------------------------------
PyTorch 1.0.0 compiled w/ CUDA 10.0
Running with Python 3.6 and CUDA 9.0.176

`pip3 list` truncated output:
numpy (1.15.4)
torch (1.0.0)
torch-salad (0.2.1a0)
torchsummary (1.5.1)
torchvision (0.2.1)
--------------------------------------------------------------------------------
  cProfile output
--------------------------------------------------------------------------------
         14086208 function calls (13987821 primitive calls) in 197.021 seconds

   Ordered by: internal time
   List reduced from 7987 to 15 due to restriction <15>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1050   89.731    0.085   89.731    0.085 {method 'cpu' of 'torch._C._TensorBase' objects}
    11550   15.540    0.001   15.540    0.001 {built-in method conv2d}
      350   10.093    0.029   10.093    0.029 {built-in method binary_cross_entropy_with_logits}
     1750    9.765    0.006    9.765    0.006 {built-in method cat}
    10150    8.467    0.001    8.467    0.001 {built-in method threshold_}
    39682    6.891    0.000    6.891    0.000 {method 'decode' of 'ImagingDecoder' objects}
      700    6.191    0.009    6.191    0.009 {method 'pin_memory' of 'torch._C._TensorBase' objects}
      345    5.894    0.017    5.894    0.017 {method 'run_backward' of 'torch._C._EngineBase' objects}
      350    3.969    0.011    5.149    0.015 C:\Users\Sklipnoty\PycharmProjects\SemanticSegmentation\unet_pytorch\Loss.py:5(dice_loss)
     1750    3.226    0.002    3.226    0.002 {built-in method torch._C._nn.upsample_bilinear2d}
     3135    2.986    0.001    2.986    0.001 {method 'convert' of 'ImagingCore' objects}
    59340    2.496    0.000    2.496    0.000 {method 'mul_' of 'torch._C._TensorBase' objects}
      700    2.383    0.003    2.383    0.003 {built-in method stack}
    59340    2.367    0.000    2.367    0.000 {method 'add_' of 'torch._C._TensorBase' objects}
      968    2.087    0.002    2.087    0.002 {method 'to' of 'torch._C._TensorBase' objects}


--------------------------------------------------------------------------------
  autograd profiler output (CPU mode)
--------------------------------------------------------------------------------
        top 15 events sorted by cpu_time_total

------  ---------------  ---------------  ---------------  ---------------  ---------------
Name           CPU time        CUDA time            Calls        CPU total       CUDA total
------  ---------------  ---------------  ---------------  ---------------  ---------------
to         298077.616us          0.000us                1     298077.616us          0.000us
to         292503.516us          0.000us                1     292503.516us          0.000us
to         292324.815us          0.000us                1     292324.815us          0.000us
to         291869.888us          0.000us                1     291869.888us          0.000us
to         286080.238us          0.000us                1     286080.238us          0.000us
to         285979.112us          0.000us                1     285979.112us          0.000us
to         284833.483us          0.000us                1     284833.483us          0.000us
to         283133.188us          0.000us                1     283133.188us          0.000us
to         282986.902us          0.000us                1     282986.902us          0.000us
to         281562.831us          0.000us                1     281562.831us          0.000us
to         281104.580us          0.000us                1     281104.580us          0.000us
to         280239.332us          0.000us                1     280239.332us          0.000us
to         280235.454us          0.000us                1     280235.454us          0.000us
to         280027.385us          0.000us                1     280027.385us          0.000us
to         279814.328us          0.000us                1     279814.328us          0.000us

--------------------------------------------------------------------------------
  autograd profiler output (CUDA mode)
--------------------------------------------------------------------------------
        top 15 events sorted by cpu_time_total

        Because the autograd profiler uses the CUDA event API,
        the CUDA time column reports approximately max(cuda_time, cpu_time).
        Please ignore this output if your code does not use CUDA.

------  ---------------  ---------------  ---------------  ---------------  ---------------
Name           CPU time        CUDA time            Calls        CPU total       CUDA total
------  ---------------  ---------------  ---------------  ---------------  ---------------
to         168797.231us        187.500us                1     168797.231us        187.500us
to         165418.527us        205.811us                1     165418.527us        205.811us
to         163756.465us        187.500us                1     163756.465us        187.500us
to         160709.951us        171.875us                1     160709.951us        171.875us
to         160394.107us        382.812us                1     160394.107us        382.812us
to         160272.480us        187.500us                1     160272.480us        187.500us
to         160098.488us        203.125us                1     160098.488us        203.125us
to         160089.345us        179.688us                1     160089.345us        179.688us
to         160002.627us        269.531us                1     160002.627us        269.531us
to         159931.977us        218.750us                1     159931.977us        218.750us
to         159812.289us        312.500us                1     159812.289us        312.500us
to         159762.973us        187.500us                1     159762.973us        187.500us
to         159516.392us        234.375us                1     159516.392us        234.375us
to         159412.496us        429.688us                1     159412.496us        429.688us
to         159323.007us        351.562us                1     159323.007us        351.562us