Oh, interesting.  Thank you so much.  Looks like it will certainly save a multiplication step.  I’m not sure what the implications on autograd that will have but hopefully some speed improvement, even though it looks like it will still consume memory where it ought not.
I believe the slight modification to your suggestion is that I need to make sure to only repeat along the batch axis.  Not repeat the single 10 nodes 10 times also.  So my final code looks like:
single_layer = torch.randn( ( 1,10))
batch = torch.randn( ( 100,5))

replicated_single = single_layer.repeat( batch.size(0),1)
input_layer = torch.cat( [ batch,replicated_single], dim=1)