Question: Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size 5000.

My implementation of boltzmann exploration:

```
def get_action(state,episode,temperature = 1):
    state_encod = np.reshape(state, [1, state_size])
    q_values = model.predict(state_encod)         
    prob_act = np.empty(len(q_values[0]))

    for i in range(len(prob_act)):
        prob_act[i] = np.exp(q_values[0][i]/temperature)

    #numpy matrix element-wise division for denominator (sum of numerators)
    prob_act = np.true_divide(prob_act,sum(prob_act))

    action_q_value = np.random.choice(q_values[0],p=prob_act)
    action_keys = np.where(q_values[0] == action_q_value)
    action_key = action_keys[0][0]
    action = index_to_action_mapping[action_key]
    return action    

```

If my temperature variable is 200, after 100 episodes I get an error 

```
ValueError: probabilities contain NaN

```

If my temperature is 1 in very few episodes i get NaN error.

Why is this happening. Am I doing something wrong here? How to select the temperature variable? Can someone help me with this. 

Thanks.

Answer: 
This is an ideal case for feature engineering!

I did this same case for myself using the google takeaway data to predict whether I am at home or at work.

Instead of just using time I extracted the following features:

1. Work Day --> 1 / 0
2. Day of the Week
3. Month
4. Year
5. Time

I then trained a random forest classification model to tell me whether I am at home, at work or other place based on those five features.

As a successive step I used this model to actually identify dates where I "moved" or was "on holiday" because of the difference between prediction and actual labels.
Choose any classifier you like and which fits your requirements...would be the "easy" answer :D

Your problem sounds as if you could go for an easy algorithm for classification up to complicated ones depending on which additional features you want to use. From the top of my head seasonal features could be interesting: Days until the next public holiday, days until christmas, vacation yes/no, month, etc.

Your output is binary which in principal could mean that you start with a Perceptron (or Multi Layer Perceptron) as a start. A neural network of your choice would be fine as well.

If you want to start with an easy to implement algorithm (for example as a benchmark for further experiments), maybe go with an easy decision tree or logistic regression.