The explanation of all logged values can be found on the documentation page of the Logger class:
eval/
All eval/ values are computed by the EvalCallback.
* mean_ep_length: Mean episode length
* mean_reward: Mean episodic reward (during evaluation)
* success_rate: Mean success rate during evaluation (1.0 means 100% success), the environment info dict must contain an is_success key to compute that value
rollout/
* ep_len_mean: Mean episode length (averaged over 100 episodes)
* ep_rew_mean: Mean episodic training reward (averaged over 100 episodes), a Monitor wrapper is required to compute that value (automatically added by make_vec_env).
* exploration_rate: Current value of the exploration rate when using DQN, it corresponds to the fraction of actions taken randomly (epsilon of the "epsilon-greedy" exploration)
* success_rate: Mean success rate during training (averaged over 100 episodes), you must pass an extra argument to the Monitor wrapper to log that value (info_keywords=("is_success",)) and provide info["is_success"]=True/False on the final step of the episode
time/
* episodes: Total number of episodes
* fps: Number of frames per seconds (includes time taken by gradient update)
* iterations: Number of iterations (data collection + policy update for A2C/PPO)
* time_elapsed: Time in seconds since the beginning of training
* total_timesteps: Total number of timesteps (steps in the environments)
train/
* actor_loss: Current value for the actor loss for off-policy algorithms
* approx_kl: approximate mean KL divergence between old and new policy (for PPO), it is an estimation of how much changes happened in the update
* clip_fraction: mean fraction of surrogate loss that was clipped (above clip_range threshold) for PPO.
* clip_range: Current value of the clipping factor for the surrogate loss of PPO
* critic_loss: Current value for the critic function loss for off-policy algorithms, usually error between value function output and TD(0), temporal difference estimate
* ent_coef: Current value of the entropy coefficient (when using SAC)
* ent_coef_loss: Current value of the entropy coefficient loss (when using SAC)
* entropy_loss: Mean value of the entropy loss (negative of the average policy entropy)
* explained_variance: Fraction of the return variance explained by the value function, see https://scikit-learn.org/stable/modules/model_evaluation.html#explained-variance-score (ev=0 => might as well have predicted zero, ev=1 => perfect prediction, ev<0 => worse than just predicting zero)
* learning_rate: Current learning rate value
* loss: Current total loss value
* n_updates: Number of gradient updates applied so far
* policy_gradient_loss: Current value of the policy gradient loss (its value does not have much meaning)
* value_loss: Current value for the value function loss for on-policy algorithms, usually error between value function output and Monte-Carle estimate (or TD(lambda) estimate)
* std: Current standard deviation of the noise when using generalized State-Dependent Exploration (gSDE) 242