Hi Sunakutra!



 Sunakutra76:

It seems that in fact the model returns the same prediction for every input of my batches during the training phase, as shown in the following test with batches of batch_size = 10 :


(As an aside, please don’t post screenshots of textual information.  It
breaks accessibility, searchability, and copy-paste.)
It works for me, at least in so far as I don’t get the same prediction for
every input.
I copied your code, made a couple of the changes you made, and I can
run it and get non-identical predictions.
Two suggestions:  You might probe the bias in your final Linear
layer.  If the output of the preceding layer gets “stuck” at zero, then
all of your final-layer predictions will just be the final layer’s `bias’.
Also, verify that your batch doesn’t consist (for some weird reason)
of 10 copies of the same input.
Here’s a script that shows non-identical predictions (for random input
data) and shows how to look at the model layers and the final bias:
import torch
from torch import nn
print (torch.__version__)

_ = torch.manual_seed (2021)

n = 20
d = 20
batch_size = 10

class NeuralNetwork(nn.Module):
    def __init__(self): ######methode dans laquelle on definit les couches du reseau
        super(NeuralNetwork, self).__init__()
        self.flatten = nn.Flatten() ########## convertit en tenseur 1D
        self.linear_relu_stack = nn.Sequential(
            # nn.Linear(d, n, bias=False),
            # nn.Linear(n, 6),
            nn.Linear(d, 6),
            nn.ReLU(),
            nn.Linear(6, 15),
            nn.ReLU(),
            nn.Linear(15, 10),
            nn.ReLU(),
            nn.Linear(10, 5),
            nn.ReLU(),
            nn.Linear(5, 1),
            # nn.ReLU()
        )
    
    def forward(self, x):
        logits = self.linear_relu_stack(x)
        return torch.reshape(logits, (-1,))

model = NeuralNetwork()

batch_of_samples = torch.randn (batch_size, d)

print ("model (batch_of_samples) = ...")
print (model (batch_of_samples))

print ("model = ...")
print (model)
print ("model._modules['linear_relu_stack'][8] =", model._modules['linear_relu_stack'][8])
print ("model._modules['linear_relu_stack'][8].bias =", model._modules['linear_relu_stack'][8].bias)

And here is its output:
1.7.1
model (batch_of_samples) = ...
tensor([0.3791, 0.4269, 0.3727, 0.4019, 0.3934, 0.4013, 0.3771, 0.3855, 0.3729,
        0.3700], grad_fn=<ViewBackward>)
model = ...
NeuralNetwork(
  (flatten): Flatten(start_dim=1, end_dim=-1)
  (linear_relu_stack): Sequential(
    (0): Linear(in_features=20, out_features=6, bias=True)
    (1): ReLU()
    (2): Linear(in_features=6, out_features=15, bias=True)
    (3): ReLU()
    (4): Linear(in_features=15, out_features=10, bias=True)
    (5): ReLU()
    (6): Linear(in_features=10, out_features=5, bias=True)
    (7): ReLU()
    (8): Linear(in_features=5, out_features=1, bias=True)
  )
)
model._modules['linear_relu_stack'][8] = Linear(in_features=5, out_features=1, bias=True)
model._modules['linear_relu_stack'][8].bias = Parameter containing:
tensor([0.2732], requires_grad=True)

If the problem persists, could you post a complete, runnable script
that sets torch.manual_seed (so we can reproduce the initialization
of your model weights), builds an input tensor from explicit values (so
that when we copy-paste your script we’ll be building the same tensor),
and prints out one batch of identical predictions?
Best.
K. Frank