I am using only 44 samples (and batch size 4) but I still can’t overfit the model when BS > 1
I resized the images to 64x64 and now the input to the FC is smaller:
class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(3, 6, 5),
            nn.ReLU(),
            nn.MaxPool2d(2, 2))
        self.layer2 = nn.Sequential(
            nn.Conv2d(6, 15, 5),
            nn.ReLU(),
            nn.MaxPool2d(2, 2))
        self.dropout = nn.Dropout(0.2)
        self.fc1 = nn.Linear(2535, 512)          
        self.fc2 = nn.Linear(512, 1)

    def forward(self, x):
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.reshape(out.size(0), -1)
        out = self.dropout(out)
        out = self.fc1(out)
        out = F.relu(out)
        out = self.dropout(out)
        out = self.fc2(out)
        return out

Unfortunately, the model keeps predicting the average value.
Do you know any examples of object counting for computer vision?
All the object counting examples I found (Surprisingly, 99% of Pytorch examples are classifiers) on the internet use a detector (such as YOLO) or density maps (for crowd counting). Maybe density maps is the way to go.
Still disappointing that I can’t solve such an easy task of counting balls with a Pytorch CNN.
Thanks for your help @ptrblck