In the most common usage, you would have no activation between
your last Linear layer and your BCEWithLogitsLoss loss criterion,
neither a Sigmoid nor a LeakyReLU .

Makes sense, considering the shape of the sigmoid adding a ReLU after the linear layer would make my lowest possible value close to 0.5.
For now, I’ve also tried with 3 FC layers and it outperforms the previous code (5-fold cross validation).
        self.input_layer = nn.Sequential(nn.Dropout(self.dropout_rate),
                                         nn.Linear(2048,hidden_sizes[0]),
                                         nn.LeakyReLU(),
                                         nn.Flatten(start_dim=-2),
                                         nn.Linear(hidden_sizes[0] * 36, hidden_sizes[1]),
                                         nn.LeakyReLU(),
                                         nn.Linear(hidden_sizes[1], num_classes)
                                        )


In a similar vein, if the 6 points in your paths are “close enough”
together, perhaps they should be treated as a sequence, and
convolved over.

The points are physically rather far and may contain dissimilar information. Think navigating in street view on google maps, but with obstructions between the navigable points. Moreover, wouldn’t that output a smaller length sequence at the output? The goal is to get a probability distribution at each time step at eval time (it’s for a down stream task).

So perhaps you should have a three-channel (up-forward-down)
Conv2d that convolves over the six points in your paths and the
twelve views in your horizontal panorama slices. Or perhaps even
a full Conv3d over all three.

Good point! I don’t think that the points are arranged like that. From what I remember the first 12 views are bottom, next 12 are middle, last 12 are top, I’ll check.  With a stride of 12 it should work. It’s getting late, but I’ll try that tomorrow and see how it performs.