Workshop track -ICLR 2018 DLVM: A MODERN COMPILER INFRASTRUCTURE FOR DEEP LEARNING SYSTEMS
Deep learning software demands reliability and performance. However, many of the existing deep learning frameworks are software libraries that act as an unsafe DSL in Python and a computation graph interpreter. We present DLVM, a design and implementation of a compiler infrastructure with a linear algebra intermediate representation, algorithmic differentiation by adjoint code generation, domainspecific optimizations and a code generator targeting GPU via LLVM. Designed as a modern compiler infrastructure inspired by LLVM, DLVM is more modular and more generic than existing deep learning compiler frameworks, and supports tensor DSLs with high expressivity. With our prototypical staged DSL embedded in Swift, we argue that the DLVM system enables a form of modular, safe and performant frameworks for deep learning.
The success of Deep Learning is, in no small part, due the development of libraries and frameworks which have made building novel models much easier, faster and less error prone and also make taking advantage of modern hardware (such as GPUs) more accessible. This is still a vital area of work, as new types of models and hardware are developed.
This work argues that prior solutions do not take advantage of the fact that a tensor compiler is, essentially, just a compiler. They introduce  DLVM (and NNKit) which comprises LLVM based compiler infrastructure and a DSL allowing the use of Swift to describe a typed tensor graph. Unusually, compared to most frameworks, gradients are calculated using source code transformation, which is argued to allow for easier optimization.
This paper is not well-adapted for an ICLR audience, many of which are not experts in compilers or LLVM. For example, the Figure 3, table 1 would be benefit from being shorter with more exposition on what the reader should understand and take away from them.
The primary weakness of this work is the lack of careful comparison with existing framework. The authors mention several philosophical arguments in favor of their approach, but is there a concrete example of an model which is cumbersome to write in an existing framework but easy here? (e.g. recent libraries pytorch, TF eager can express conditional logic much more simply than previous approaches, its easy to communicate why you might use them). Because of this work seems likely to be of limited interest to the ICLR audience, most of which are potentially interested users rather than compiler experts. There is also no benchmarking, which is at odds with the claims the compiler approaches allows easier optimization.
One aspect that seemed under-addressed and which often a crucial aspect of a good framework, is how general purpose code e.g. for loading data or logging interacts with the accelerated tensor code.