I faced the same problems with the new 3090 card and cuda 11.1 trying to install pytorch. I followed the directions here 135 to compile pytorch from source. I was able to do this within a conda environment also.