We introduce the 2-simplicial Transformer, an extension of the Transformer which includes a form of higher-dimensional attention generalising the dot-product attention, and uses this attention to update entity representations with tensor products of value vectors. We show that this architecture is a useful inductive bias for logical reasoning in the context of deep reinforcement learning.
CONTRIBUTIONS:
C1. Simplicialization of attention. Interpreting standard attention weights of a head as the model’s estimate of the probability of an edge = 1-simplex linking the variables encoded by 2 blocks of the Transformer, representing that the blocks stand in a binary relation encoded in the head, a generalization to 2-simplexes is made: now attention also estimates the probability of a 2-simplex indicating that three blocks stand in an arity-3 relation. 
C2. 2-simplicial attention. The standard query-key matching function, the scalar (dot) product, is related to the area of the 2-simplex determined by the 2 vectors and the origin, and this is generalized to the (unsigned) scalar triple product <a,b,c>, analogously related to the volume of the 3-simplex determined by 3 vectors and the origin. This now serves as the matching function between a query and 2 keys. Each head in each block generates a value vector u and two key vectors k1, k2 and the weight of attention from block(i) (with query p(i)) to the ordered pair (block(j), block(k)), a(i,j,k), is a softmax over <p(i), k1(j), k2(k)>. Attention returns to block(i) a sum in which a(I,j,k) weights B(u(j)*u(k)), with B a learned linear map and * the tensor product. 
C3. Experimental results applying Transformers T1 (with standard 1-simplicial attention) and T2 (with new 2-simplicial attention) to modeling an agent in bridge BoxWorld, trained with deep RL. This game crucially involves 3-way entity interactions, as keys of 2 colors open a box yielding a key of a 3rd color. T2 learns significantly faster than T1 (in the sense that the 1-standard-deviation-neighborhood of the learning curve of T2 becomes better than that of T1, plotted against environmental steps: Fig.4, and also essentially so when plotted against time adjusted steps: Fig 5).
RATING: Accept
REASONS FOR RATING (SUMMARY). Generalizing attention from 2nd- to 3rd-order relations is an important upgrade, and the mathematical context in which this is derived is insightful and may lead to further progress in the development of Transformers capable of constructing still richer structures. The experiments yield clear evidence of the value of 3rd-order attention in the context of a game designed to highlight 3rd-order relations.
Strengths
The exposition is clear and situated in a rather sophisticated formal setting. The connection to Clifford algebras may yield further fruit, besides the scalar triple product that is crucial to the definition of 2-simplicial attention. Although I am not an expert in RL, the experiments reported seem sound and the results clear. I believe that the strength of the paper justifies its length of 8.5 pages: the exposition of the key ideas, for this reader, hits a sweet spot between overly concise and overly verbose, and the ideas call for the quantity of space devoted to them. The decisions of what material to place in the Appendix seem well made. Although I have not studied the entire (13-page) Appendix, what I have read is clear and enlightening, another major contribution of the paper.
Weaknesses
Future work testing the value of 3rd-order attention in tasks that are less clearly perfectly designed for it will substantially strengthen the case for it. But in my view it is right to start testing a new architecture by showing it can indeed do what it is designed to do; further tests showing that what it is designed to do is of general utility are a second step. In this case, the first step, including creating of the model itself (consuming 5 non-verbose pages), is substantial enough to warrant publication.
In my good-quality printout of the paper, I can’t see curves for best runs in Fig. 4.