If I remove all the GPipe stuff it works. I took out
partitions = torch.cuda.device_count()
	sample = torch.rand(64, 1, 227, 227)
	balance = balance_by_time(partitions, model, sample)
	model = GPipe(model, balance, chunks=8)

as well as
data = data.to(model.devices[0], non_blocking=True)
target = target.to(model.devices[-1], non_blocking=True)

in both the train and test functions