This is a solution (hacky) that does work
class DropNames:
    def __init__(self):
        pass

    def __call__(self, sample):
        new_sample = {}

        for k, v in sample.items():
            if isinstance(v, torch.Tensor) and any(v.names):
                new_sample[k + '_names'] = ';'.join(v.names)  # collate_fn will do funky things without this.
                v = v.rename(None)
            new_sample[k] = v

        return new_sample


class AddNames:
    def __init__(self, add_batch_dimension=True):
        self.add_batch_dimension = add_batch_dimension

    def __call__(self, sample):
        names = [_[:-6] for _ in sample.keys() if _[-5:] == 'names']
        new_sample = {k: v for k, v in sample.items() if k[-5:] != 'names'}

        for name in names:
            if self.add_batch_dimension:
                names = ['batch'] + sample[name + '_names'][0].split(';')

            else:
                names = sample[name + '_names'].split(';')

            new_sample[name] = sample[name].rename(*names)

        return new_sample

The first function can be added to the end of all the transforms (which are typically composed). This will drop the names, create string out of it separated by ; (assuming your variables have no ;).
The reason I do this, is because the collate function can do weird things with a list, and if you use the second function as:
add_names = AddNames(add_batch_dimension=True)
    for iter_idx, data in enumerate(data_loader):
    data = add_names(data)

you get the names in the tensors back. You might need to drop them again later, as many operations do not  yet support this, but in my case, I make a lot of use of the FFT which in the current implementation requires you to permute the axis often. Having Named Tensors really helps in reducing bugs.
So the above works with pytorch 1.5, so perhaps it is of use to anyone. Improvements are definitely welcome.