  The recent success of Transformer in natural language processing has sparked
its use in various domains. In offline reinforcement learning (RL), Decision
Transformer (DT) is emerging as a promising model based on Transformer.
However, we discovered that the attention module of DT is not appropriate to
capture the inherent local dependence pattern in trajectories of RL modeled as
a Markov decision process. To overcome the limitations of DT, we propose a
novel action sequence predictor, named Decision ConvFormer (DC), based on the
architecture of MetaFormer, which is a general structure to process multiple
entities in parallel and understand the interrelationship among the multiple
entities. DC employs local convolution filtering as the token mixer and can
effectively capture the inherent local associations of the RL dataset. In
extensive experiments, DC achieved state-of-the-art performance across various
standard RL benchmarks while requiring fewer resources. Furthermore, we show
that DC better understands the underlying meaning in data and exhibits enhanced
generalization capability.

Machine Learning