TF still has a single compute stream (transfers and NCCL ops each use a different stream) per session. This is not trivial to change because the GPU memory allocator implicitly synchronizes all allocations on that compute stream.
We are implementing multi-stream support in the HLO to TFRT compiler, but that’s not ready yet.