The guy who ran the tests said that he couldn’t get any video output from the card (and he tried a few different motherboards/video cables).
So he didn’t run any specific testing/benchmarking.
Also I didn’t mentioned that I originally bought this card from a miner, so he might have burned a few transistors here and there 
With that said, the card is listed in nvidia-smi, and I can get it to run some models.
With my latest testing, the card seems to handle a model composed of just RNNs and a Wavenet model just fine. (I used this wavenet model: https://github.com/vincentherrmann/pytorch-wavenet/blob/master/wavenet_model.py 2, and built-in torch GRUs)
It seem to have issues with my TCN model (taken here: https://github.com/locuslab/TCN/blob/master/TCN/tcn.py 1)
It’s very hard to debug because it usually fails after 5-20min of training, so I’m never sure of anything.
There is no clear pattern about what in the model (batch norm layers, regularization, convs, concats, indexing…) is causing the GPU to fail. And I did not have the patience to figure it out.
I think I’ll just get a brand new 2080TI and this will be it.
Thanks for jumping in, I appreciate it