Dear albanD, thank you for your feedback. I have already read something about the TensorDataset / DataLoader, I would like to look at it again.
To be able to use my data in a DDQN reinforcement learner, I programmed my own environment (it uses Numpy). I do NOT use Cartpole / MountainCar environment or anything.
The big question is:
Can I increase the speed at which PyTorch works by a factor of > 2-3 by loading the data into tensors? That’s what i need to be at the same speed as with Keras/Theano. I have to rewrite the complete workflow / environment to do this. I have doubts, do you have any experience with the topic?
See the following, simple example:
MAIN LOOP (everything else removed)
    while not done:
        #////////////////////////////////////////////////
        SYS.ENV['STEPS'] += 1
        #////////////////////////////////////////////////
        action = SYS.AGT['CALL'].get_action(SYS, state)
        #////////////////////////////////////////////////
        next_state, reward, done, info = SYS.ENV['CALL'].step(action)
        #////////////////////////////////////////////////
        SYS.AGT['CALL'].remember(state, action, reward, next_state, done) # add data to memory
        #////////////////////////////////////////////////
        state = deepcopy(next_state) # copy
        #////////////////////////////////////////////////
        tqdm_e.update(1)
        #////////////////////////////////////////////////
        if(done): return(True)

The important part here is SYS.AGT[‘CALL’].get_action(SYS, state), it “predicts” an action by a state.
I can do this with Keras/Theano backend:
    def get_action(self, SYS, state): # (Keras/Theano)
        #////////////////////////////////////////////////
        self.state = convert_1D2D(state, 1, self.state_size)
        #////////////////////////////////////////////////
        qvals = self.DQN.predict(self.state)
        #////////////////////////////////////////////////
        self.action = np.argmax(qvals[0])
        #////////////////////////////////////////////////
        return(self.action)

ONE simple predict per call: [00:01<00:00, 1635.84 item(s)/s]
And i can do this with PyTorch:
    def get_action(self, SYS, state): # (PyTorch)
        #////////////////////////////////////////////////
        self.state = torch.from_numpy(state).float().unsqueeze(0)
        #////////////////////////////////////////////////
        self.DQN.model.eval()
        #////////////////////////////////////////////////
        with torch.no_grad():
            qvals = self.DQN.predict(self.state).numpy() # this predict is forward or model(x)
        #////////////////////////////////////////////////
        self.action = np.argmax(qvals[0])
        #////////////////////////////////////////////////
        return(self.action)

ONE simple forward per call: [00:03<00:00, 643.33 item(s)/s]
1635÷643 = ~ 2.5
That means, Keras with the Theano backend is about 2.5 times faster for this simple task. Both are using 1 Cpu Core at the moment. I would turn on multicore support as the last thing. Before that, it must be clear which data, where and how is processed. Thanks anyway for the hint.
Do you think I can make up the difference by using PyTorch Tensors for my data?
Can anyone tell me something about my question 2?