Since i didnt not manage too make it run with CrossEntropyLoss i simply just tryed another lossfct.
With that i trained the vrn-network 5000 times with a single picture and the model.
However i thought regardless of lossfct etc. i can always overfit a model too fit it.
This is my script :
VRN = vrn.vrn_unguided
VRN.load_state_dict(torch.load('vrn-unguided-selfmade.pth'))
# telling the model it will be trained
VRN.train()
# lossfct to calculate error while training -> sigmoid cross entropy loss  in original vrn.
lossfct = torch.nn.modules.loss.MSELoss()
# optimizer settings
optimizer = torch.optim.SGD(VRN.parameters(), lr=1)

# this is for data loading. it is checked and loading correctly
voxel_array = np.zeros((1, 384, 384, 384), dtype=float)
file = open("sample_1.txt","r")
for line in file:
    if not "#" in line:
        x = line.partition(" ")[0]
        y = line.partition(" ")[2].partition(" ")[0]
        z = line.partition(" ")[2].partition(" ")[2].partition(" ")[0]
        voxel_array[0, int(x), int(y), int(z)] = 1
file.close()
# image to model. 
image_file = 'rsz_pic_zu_sample.jpg'
image = cv.imread(image_file)
try:
    image_height, image_width, image_depth = image.shape
except:
    print('cannot load image:', image_file)
# vrn output
inp = torch.from_numpy(image.transpose((2, 0, 1))).float().unsqueeze_(0)

voxel_array = torch.from_numpy(voxel_array)
voxel_array = voxel_array.float()

for epoch in range(10):
    print(epoch)
    VRN.train()
    predicted_model = VRN(inp)[-1].cpu()
    # vol = predicted_model.numpy()
    # vol = predicted_model.numpy()

    optimizer.zero_grad()

    # voxel_array = voxel_array.squeeze(0)

    # vol = torch.from_numpy(vol)
    loss = lossfct(predicted_model.squeeze(0), voxel_array.squeeze(0))
    print(loss)
    loss.backward()
    optimizer.step()

torch.save(VRN.state_dict(), "vrn-unguided-selfmade.pth")
print("Done, saved CNN")

However after running this 5k times.
If i test the network with the exact same picture as in training, the result is way different then the training-model.
This is the model trained on

model.PNG458×556 27.8 KB

and this is the output after 5k training on the same data

Bild_2021-04-14_224917532×616 24.7 KB

How can it be, that after so many iterations, even the train data gives such a different result?