Hello Markus!
I haven’t looked at your code in detail, but here are comments on some
things I noticed:



 CesMak:

class my_model(nn.Module):
    def __init__(self, n_in=180, n_hidden=1, n_out=60):
        super(my_model, self).__init__()
        self.n_in  = n_in
        self.n_out = n_out

        self.fc1 = nn.Linear(self.n_in, 120)
        self.relu1 = nn.ReLU(inplace=True)
        self.fc2 = nn.Linear(120, self.n_out)



First note that this version of my_model only supports one hidden
layer.  So I would just get rid of the n_hidden argument to the
constructor – it’s not used at all, so it just confuses things.
It’s not a big deal, but you might consider not hard-coding the value
of 120 for the number of hidden neurons.  It’s a value you might
want to play with, so you could make it an argument to your
my_model constructor, e.g.,
    def __init__(self, n_in=180, n_middle=120, n_out=60):



    optimizer = torch.optim.Adam(model.parameters(), lr=1e-3, weight_decay=1e-4)



Adam is a perfectly reasonable optimizer, and often works better
than SGD.  But you might also want to experiment with SGD as it
has fewer moving parts, so it can make things a little easier to
understand when you’re developing a new model.


        #TODO recast target  one hot to single!



Indeed.


        pred = model(data)
        print(pred.shape, target.shape) # torch.Size([1, 60]) torch.Size([1, 1, 61])

        #ValueError: Expected input batch_size (1) to match target batch_size (61).
        # Fails:
        loss = criterium(pred, target.view(-1))



Several things here:
First (and not so important), your batch size – the number of
individual input samples you process per optimizer step – is 1.
As a general rule (and, as a general rule, general rules for neural
networks are often wrong),  you want to have “many” samples
in a batch.  I don’t really have any intuition for your specific problem,
but I think playing with batch sizes ranging from 50 to 500 might
make sense.  (Powers of 2 look cool.)
Now for the error:  Your target shape is wrong.  data.shape is
presumably [nBatch, 180] (where you have nBatch = 1).
pred.shape = [nBatch, 60] is correct.
For CrossEntropyLoss you need to have target.shape = [nBatch],
that is, a single integer class label for each sample in your batch.
target.shape = [1, 1, 61]is wrong, and taking .view (-1)
doesn’t fix that because target.view (-1).shape = [61], which
is still wrong.
As I understand it, when you read in target, you read in 61 numbers:
a length-60 one-hot encoded class label, followed by what you call
bestq.  First, you need to get rid of bestq – your model has no way
to know to ignore it, so if you include it you will get an error.  Second,
you need to convert your one-hot encoded (59 '0’s and one ‘1’)
class-label vector into a single integer (in the range [0, 59], inclusive)
class label.  You can do this as follows:
dummy, integer_class_batch = one_hot_class_batch.max (dim = 1)`

This version of max() returns the tuple (max, argmax).
dummy = max = 1, and we discard it.  argmax is how we get the
single integer class label from the one-hot vector.
Here, one_hot_class_batch.shape = [nBatch, 60], and
integer_class_batch.shape = [nBatch].


Further Notes:
I have currently 18786 batches -> I think I need more or?


I don’t have intuition about your specific problem.  In general,
20,000 samples for a (straightforward) 60-class classification
problem should be plenty.  But the structure of your card game
could make the problem hard.  How easy or hard is it for a
person to choose the right move just looking at the hands?
A couple more comments:
You might consider first building a similar model for a well-studied
problem (for example MNIST digit classification).  That way you’ll
be able to separate bugs in your code (or suboptimal models) from
the problem itself being hard to solve.
As a modified approach, you might consider training your model not
with a target that is the single best move, but with scores for how
good each possible move is (or, for practical purposes, perhaps how
good each of the ten best moves is).
Suppose you have five possible moves: (bad, bad, good, good, best).
Categorical cross-entropy (that is, target is the single correct class
label) will score the two following predictions equally:
   (33% (bad), 33% (bad),  0% (good),  0% (good), 33% (best))
   ( 0% (bad),  0% (bad), 33% (good), 33% (good), 33% (best))

But a model that makes the second prediction will play a much better
game (even if not perfect) than a model that makes the first.
You talk about bestq.  If you have q values for all possible moves
(or for the best several moves), you could use (values derived from)
your q values as your target, and use what I’ll call (for lack of a
better term) “full” cross-entropy for your loss function.
Last, you mention:


Using a constraint is a cool idea (unfortunately I am not sure how to incooperate that).


I think you might be referring to the constraint I speculated about that
you can only play a card you actually have in your current hand.
My point was not that you would somehow impose such a constraint.
(That is, you wouldn’t ever make such a constraint explicit or build it
into your model.)  But if such a constraint is part of the rules of your
game, your model with have to learn that rule.  My suggestion to add
to the model a “late look” at what’s in the current hand is a proposal
for making it easier for the model to learn that rule.
Good luck!
K. Frank