I am experiencing a really strange issue with out of memory errors on pytorch:
$ python -c "import torch; torch.randn(12,12).cuda()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
RuntimeError: CUDA error: out of memory

Background information:
I just created a new environment with the following (from the documentation):
conda create -n pytorch-test
conda install -n pytorch-test -c pytorch pytorch cudatoolkit=11.0

And my GPU:
$ echo $CUDA_VISIBLE_DEVICES                                                                                           
2
$ nvidia-smi -i 2                                                                                                      
Thu Nov 19 12:52:56 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.38       Driver Version: 455.38       CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   2  Tesla P100-PCIE...  Off  | 00000000:0E:00.0 Off |                    2 |
| N/A   47C    P0    29W / 250W |      2MiB / 16280MiB |      0%   E. Process |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Any help is much appreciated, I kinda desperate here 
[EDIT]: This seems to be more of a hardware issue. I am experiencing the same problem with a new installation of tensorflow, following the exact same steps… Still, if you have any pointer to solve this, I take it