I was able to use dataparallel on my model without any apparent errors. However, the performance was actually worse; which makes me think that it’s not actually using multiple gpus.

DataParallel is single-process-multi-thread data parallelism, and it replicates the input module in every forward pass, which is expected to be slow, but this is a very convenient entry point for enabling parallelism. You don’t need to do anything to enable that, and it should work fine if the batch size is large enough (to shadow the model replicating overhead)

Why am I able to use multiple gpus in tensorflow on a windows system

We are working on using libuv to enable that, as @pietern did for Windows, but timeline is TBD