Not much response so far, but I won’t give up hope.
I encountered another problem. I wanted to test the performance after training the network and unfortunately, i always get a reward of zero.
The validation loop is implemented in the train() function and looks like follows:
def train(n_games, lr):
    env = TicTacToe()
    brain = Agent(gamma=0.99, epsilon=1.0, batch_size=512, n_actions=9,
                  input_dims=[10], lr=lr)
    scores = []
    test_scores = []
    eps_history = []

    for i in range(n_games):
        if i % 100 == 0 and i > 0:
            avg_score = np.mean(scores[max(0, i-100):(i+1)])
            print('episode:', i, 'average score %.3f:' % avg_score, 'epsilon:', brain.epsilon)
        score = 0
        eps_history.append(brain.epsilon)
        observation = env.reset()
        done = False
        while not done:
            action = brain.choose_action(observation)
            observation_, reward, done, info = env.step(action)
            score += reward
            brain.store_transition(observation, action, reward, observation_, done)
            brain.learn()
            observation = observation_

        scores.append(score)

    # save model
    torch.save(brain.Q_eval.state_dict(), 'parameters.pth')

    for i in range(100):
        brain.epsilon = 0.
        score = 0
        observation = env.reset()
        done = False
        while not done:
            action = brain.choose_action(observation)
            observation_, reward, done, info = env.step(action)
            score += reward
            observation = observation_
        test_scores.append(score)

    print(np.mean(test_scores))

So while the training ended up with an average score of about 0.3, the test with the trained network resulted in a score of 0. This changes however, when I add the two lines:
brain.store_transition(observation, action, reward, observation_, done)
brain.learn()

That way, I get similar results as during the end of the training loop. But that way I train the network further, which is not what I want… Any advice why this happens?