Shalev-Shwartz, Amnon Shashua, Moshe Tenenholtz

AI21 Labs

May 3, 2022

Abstract
Huge language models (LMs) have ushered in a new era for AI, serving as a gate-
way to natural-language-based knowledge tasks. Although an essential element of
modern AI, LMs are also inherently limited in a number of ways. We discuss these
limitations and how they can be avoided by adopting a systems approach. Con-
ceptualizing the challenge as one that involves knowledge and reasoning in addition
to linguistic processing, we deﬁne a ﬂexible architecture with multiple neural mod-
els, complemented by discrete knowledge and reasoning modules. We describe this
neuro-symbolic architecture, dubbed the Modular Reasoning, Knowledge and Lan-
guage (MRKL, pronounced “miracle”) system, some of the technical challenges in
implementing it, and Jurassic-X, AI21 Labs’ MRKL system implementation.

Introduction
MRKL Systems