<p>So, both the last return values of reset() and step() are the <code>info</code> element, which should be a a dictionary, not a float/int (which I used as a placeholder). Replacing this with a dummy dict fixes the problem.</p>

CC BY-SA 4.0
2024-08-16T14:03:08.440Z
2024-08-16T14:03:08.440Z
DominikS
<p>I want to train my Gymnasium model using stable-baselines3 PPO. I have used the structure below before, however for my particular problem, I keep getting the following problem when calling PPO.train():</p>
<pre><code>File ~\.conda\envs\reinforce\Lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py:65, in DummyVecEnv.step_wait(self)
 62 self.buf_dones[env_idx] = terminated or truncated
 63 # See https://github.com/openai/gym/issues/3102
 64 # Gym 0.26 introduces a breaking change
 ---&gt; 65 self.buf_infos[env_idx][&quot;TimeLimit.truncated&quot;] = truncated and not terminated
 67 if self.buf_dones[env_idx]:
 68     # save final observation where user can get it, then reset
 69     self.buf_infos[env_idx][&quot;terminal_observation&quot;] = obs

 TypeError: 'int' object does not support item assignment
</code></pre>
<p>Since it has pretty much the same structure as the classes I have used before, I am surprised why I get an error, I can't find the mistake. Does anyone have more insights?</p>
<p>The key structure of the my Gym class is as follows:</p>
<pre><code>class Cells(gym.Env):

    def __init__(
        self,
        penalty_factor=10.0,
        action_space=spaces.Box(
            low=np.array([0.0, 0.0, 0.0]),  # minima for delta mu_i
            high=np.array([1.0, 1.0, 1.0]),  # maxima for delta mu_i
            dtype=np.float64,
        ),  # action space
        observation_space=spaces.Dict(
            spaces={
                &quot;mu_1&quot;: spaces.Box(0.0, 1.0),
                &quot;mu_2&quot;: spaces.Box(0.0, 1.0),
                &quot;mu_3&quot;: spaces.Box(0.0, 1.0),
                &quot;d_1&quot;: spaces.Box(-5.0, 5.0),
                &quot;d_2&quot;: spaces.Box(-5.0, 5.0),
            }
        ),  # state space
        T_max=100,
    ):
        self.action_space = action_space
        self.observation_space = observation_space
</code></pre>
<p>....</p>
<pre><code>    def get_state(self):
        return {
            &quot;mu_1&quot;: np.array([self.mu_1], dtype=np.float64),
            &quot;mu_2&quot;: np.array([self.mu_2], dtype=np.float64),
            ...
        }

    def get_done(self):
        if self.t &gt;= self.T_max:
            return True, False
        else:
            return False, False

    def reset(self, **kwargs):
        self.t = 0
        self.mu_1 = 0
        ...

        return self.get_state(), 0.0
</code></pre>
<p>...</p>
<pre><code>def step(self, action):
        
        delta_mu_1, delta_mu_2, delta_mu_3 = action
        self.t += self.dt

        self.mu_1 += delta_mu_1 * self.dt
        self.mu_2 += delta_mu_2 * self.dt
        self.mu_3 += delta_mu_3 * self.dt
        
        # reward
        reward = self.get_reward()

        terminated, truncated = self.get_done()
        if terminated:
            state, _ = self.reset()
        else:
            state = self.get_state()
        return state, reward, terminated, truncated, 0
</code></pre>
<p>The first call of step() during training phase completes successfully, and when I print out the return values of step(), I get sensible answers. So I am pretty sure it is not an error in step() itself.</p>

<gym><stable-baselines>
Stable Baselines: 'int' object does not support item assignment
DominikS
# Stable Baselines: 'int' object does not support item assignment

(Asked by: username_0 on 2024-08-16)

I want to train my Gymnasium model using stable-baselines3 PPO. I have used the structure below before, however for my particular problem, I keep getting the following problem when calling PPO.train():
File ~\.conda\envs\reinforce\Lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py:65, in DummyVecEnv.step_wait(self)
 62 self.buf_dones[env_idx] = terminated or truncated
 63 # See https://github.com/openai/gym/issues/3102
 64 # Gym 0.26 introduces a breaking change
 ---> 65 self.buf_infos[env_idx]["TimeLimit.truncated"] = truncated and not terminated
 67 if self.buf_dones[env_idx]:
 68     # save final observation where user can get it, then reset
 69     self.buf_infos[env_idx]["terminal_observation"] = obs

 TypeError: 'int' object does not support item assignment

Since it has pretty much the same structure as the classes I have used before, I am surprised why I get an error, I can't find the mistake. Does anyone have more insights?
The key structure of the my Gym class is as follows:
class Cells(gym.Env):

    def __init__(
        self,
        penalty_factor=10.0,
        action_space=spaces.Box(
            low=np.array([0.0, 0.0, 0.0]),  # minima for delta mu_i
            high=np.array([1.0, 1.0, 1.0]),  # maxima for delta mu_i
            dtype=np.float64,
        ),  # action space
        observation_space=spaces.Dict(
            spaces={
                "mu_1": spaces.Box(0.0, 1.0),
                "mu_2": spaces.Box(0.0, 1.0),
                "mu_3": spaces.Box(0.0, 1.0),
                "d_1": spaces.Box(-5.0, 5.0),
                "d_2": spaces.Box(-5.0, 5.0),
            }
        ),  # state space
        T_max=100,
    ):
        self.action_space = action_space
        self.observation_space = observation_space

....
    def get_state(self):
        return {
            "mu_1": np.array([self.mu_1], dtype=np.float64),
            "mu_2": np.array([self.mu_2], dtype=np.float64),
            ...
        }

    def get_done(self):
        if self.t >= self.T_max:
            return True, False
        else:
            return False, False

    def reset(self, **kwargs):
        self.t = 0
        self.mu_1 = 0
        ...

        return self.get_state(), 0.0

...
def step(self, action):
        
        delta_mu_1, delta_mu_2, delta_mu_3 = action
        self.t += self.dt

        self.mu_1 += delta_mu_1 * self.dt
        self.mu_2 += delta_mu_2 * self.dt
        self.mu_3 += delta_mu_3 * self.dt
        
        # reward
        reward = self.get_reward()

        terminated, truncated = self.get_done()
        if terminated:
            state, _ = self.reset()
        else:
            state = self.get_state()
        return state, reward, terminated, truncated, 0

The first call of step() during training phase completes successfully, and when I print out the return values of step(), I get sensible answers. So I am pretty sure it is not an error in step() itself.

**Tags:** gym, stable-baselines


---

## 1 answers

### Answer by username_0 on 2024-08-16. Score: 0

So, both the last return values of reset() and step() are the `info` element, which should be a a dictionary, not a float/int (which I used as a placeholder). Replacing this with a dummy dict fixes the problem.


---

