First of all i would like to thank you for your support!!!
Here is how I normalized both features and targets, which are tensors of tensors. The goal was to normalize all values with the same inner tensor position (i.e. for each column), based on the global maximum and minimum of that position (“column”).
featuresMax = torch.max(features, dim=0)[0]
featuresMin = torch.min(features, dim=0)[0]

for i in range(0, features.shape[1]):
    for j in range(0, len(features)):
        features[j][i] = (features[j][i] - featuresMin[i])/(featuresMax[i]-featuresMin[i]) # range 0 to 1

targetMax = torch.max(targets, dim=0)[0]
targetMin = torch.min(targets, dim=0)[0]

for i in range(0, len(targets)):
    targets[i] = (targets[i] - targetMin)/(targetMax-targetMin) # range 0 to 1

The training function
model.train()
epochs = 5
errors = []
for epoch in range(epochs):
    optimizer.zero_grad() # sets gradients to zero
    for feature, target in trainset:
        y_pred = model.forward(feature)
        loss = criterion(y_pred.squeeze(), target)
        errors.append(loss.item())
        print('Epoch {}: train loss: {} MaxTarget: {} MinTarget: {} maxFeature: {} MinFeature: {}'.format(epoch, loss.item(), amax(target), amin(target), amax(feature), amin(feature)))
        # print('Epoch {}: train loss: {} '.format(epoch, loss.item()))
        loss.backward()
        optimizer.step()

The following plot shows the plt.plot(errors) of all iterations of 5 epochs (len(trainset)=2804) and does not look good because the errors/losses do the opposite of decreasing.

Unbenannt712×422 18.7 KB

I tried changing the following:

removing features (rather invariant ones)
removing hidden layers
changing the learning rate
reducing the amount of neurons of the hidden layers
changing the batch size

Chaning the batch size had the largest impact in the loss calculation (same amount of features, batch size increased from 10 to 1000, epoch increased from 5 to 50)

Questions:
Am I heading in the right direction - does the model “work as is” and it is “just” a matter of tweaking the parameters (activation functions, lr, etc.) and sets of features?
Is it plausible that the changed batch size has such an (positive) impact?
Is there a rule of thumb with regard to batch size, epoch size and sample size?