Hi everyone. I’m implementing an RL algorithm and I’m masking unavailable actions by setting their Q-values really low, so when I apply max() I never select them. Usually, I do that with float("-inf") in pure Python. The thing is, in all my experiments, I’m getting a RuntimeError: CUDA out of memory in the lines where I use this trick, like the example below:
live_next_q_vals = current_q_vals.clone().detach()[:, 1:]
live_next_q_vals[~av_actions_batch[:, 1:]] = float('-inf')

Does anyone know if using float("inf") or float("-inf") is not recommended with PyTorch tensors? I saw other implementations use large negative numbers, like -999999, but I didn’t see a reason to do that and it looks like I made the wrong choice, lol.