My guess is something is strange about trying to coalesce y across the “batch” dimension when it is getting passed back. Does working around this by removing the first dimension of y (e.g., y = y.squeeze(0)) before passing it to the net work?