  In Reinforcement Learning (RL), Laplacian Representation (LapRep) is a
task-agnostic state representation that encodes the geometry of the
environment. A desirable property of LapRep stated in prior works is that the
Euclidean distance in the LapRep space roughly reflects the reachability
between states, which motivates the usage of this distance for reward shaping.
However, we find that LapRep does not necessarily have this property in
general: two states having small distance under LapRep can actually be far away
in the environment. Such mismatch would impede the learning process in reward
shaping. To fix this issue, we introduce a Reachability-Aware Laplacian
Representation (RA-LapRep), by properly scaling each dimension of LapRep.
Despite the simplicity, we demonstrate that RA-LapRep can better capture the
inter-state reachability as compared to LapRep, through both theoretical
explanations and experimental results. Additionally, we show that this
improvement yields a significant boost in reward shaping performance and also
benefits bottleneck state discovery.

Machine Learning;Artificial Intelligence