I would expect that pytorch distributed should recognize this situation, and return the thread to the thread pool while it is waiting on the RPC.

Unfortunately that’s not something that can easily be achieved. It can be done through Python’s asyncio package but that’s not supported by PyTorch (for historical reasons I guess, and it’s hard now to retrofit it). Any other alternative would basically consist in reimplementing our own asyncio, and that’s just unreasonable.

What could I change in my code so that it “returns the thread” to the thread pool, but continues executing once the RPC completes?

I think your situation could be addressed by using the async_execution decorator, and futures, as @mrshenli already mentioned. More concretely, here is what such a code could look like:
# Imagine this is the original code:
def my_function(a):
    b = foo(a)
    c = rpc.rpc_sync("other_worker", my_other_function, args=(b,))
    d = bar(c)
    return d

# It could become like this:
@rpc.functions.async_execution
def my_function(a):
    b = foo(a)
    fut_c = rpc.rpc_async("other_worker", my_other_function, args=(b,))
    fut_d = fut_c.then(lambda fut: bar(fut.value()))
    return fut_d