(e.g. “0” for left, “1” for right) can seemingly affect the ability of an LLM to improve trajectories in the
online setting. For example, we observed that if “0” is included in the action space, LLMs may “default”
to sampling “0” (likely due to token-specific priors). Therefore, for our experiments, we use 1-indexed
integer action representations, which appears to alleviate the bias towards choosing a particular action. The
fact that action representation can sometimes affect performance complements our observations in the
Sequence Transformation section, in which we find that token mapping invariance holds to some extent,
but not entirely.
LargeLanguageModelsasGeneralPatternMachines