We present Predicted Variables (PVars), an approach to making machine learning (ML) a first class citizen in programming languages. There is a growing divide in approaches to building systems: using human experts (e.g. programming) on the one hand, and using behavior learned from data (e.g. ML) on the other hand. PVars aim to make using ML in programming easier by hybridizing the two. We leverage the existing concept of variables and create a new type, a predicted variable. PVars are akin to native variables with one important distinction: PVars determine their value using ML when evaluated. We describe PVars and their interface, how they can be used in programming, and demonstrate the feasibility of our approach on three algorithmic problems: binary search, QuickSort, and caches. We show experimentally that PVars are able to improve over the commonly used heuristics and lead to a better performance than the original algorithms. As opposed to previous work applying ML to algorithmic problems, PVars have the advantage that they can be used within the existing frameworks and do not require the existing domain knowledge to be replaced. PVars allow for a seamless integration of ML into existing systems and algorithms. Our PVars implementation currently relies on standard Reinforcement Learning (RL) methods. To learn faster, PVars use the heuristic function, which they are replacing, as an initial function. We show that PVars quickly pick up the behavior of the initial function and then improve performance beyond that without ever performing substantially worse -allowing for a safe deployment in critical applications.
This paper proposes using predicted variables(PVars) - variables that learn
their values through reinforcement learning (using observed values and
rewards provided explicitly by the programmer). PVars are meant to replace
variables that are computed using heuristics.
Pros:
* Interesting/intriguing idea
* Applicability discussed through 3 different examples
Cons:
* Gaps in explanation
* Exaggerated claims
* Problems inherent to the proposed technique are not properly addressed, brushed off as if unimportant
The idea of PVars is potentially interesting and worth exploring; that
being said, the paper in its current form is not ready for
publication.
Some criticism/suggestions for improvement:
While the idea may be appealing and worth studying, the paper does not address several problems inherent to the technique, such as:
- overheads (computational cost for inference, not only in
  prediction/inference time but also all resources necessary to run
  the RL algorithm; what is the memory footprint of running the RL?)
- reproducibility
- programming overhead: I personally do not buy that this technique -
  at least as presented in this paper - is as easy as "if statements"
  (as stated in the paper) or will help ML become mainstream in
  programming. I think the programmer needs to understand the
  underpinnings of the PVars to be able to meaningfully provide
  observations and rewards, in addition to the domain specific
  knowledge. In fact, as the paper describes, there is a strong
  interplay between the problem setting/domain and how the rewards should be
  designed.
- applicability: when and where such a technique makes sense
The interface for PVars is not entirely clear, in particular the
meaning of "observations" and "rewards" do not come natural to
programmers unless they are exposed to a RL setting. Section 2 could
provide more details such that it would read as a tutorial on
PVars. If regular programmers read that section, not sure they
understand right away how to use PVars. The intent behind PVars
becomes clearer throughout the examples that follow.
It was not always clear when PVars use the "initialization function"
as a backup solution. In fact, not sure "initialization" is the right
term, it behaves almost like an "alternative" prediction/safety net.
The examples would benefit from showing the initialization of the PVars.
The paper would improve if the claims would be toned down, the
limitations properly addressed and discussed and the implications of
the technique honestly described. I also think discussing the
applicability of the technique beyond the 3 examples presented needs
to be conveyed, specially given the "performance" of the technique
(several episodes are needed to achieve good performance).
While not equivalent, I think papers from approximate computing (and
perhaps even probabilistic programming) could be cited in the related
work. In fact, for an example of how "non-mainstream" ideas can be
proposed for programming languages (and explained in a scientific
publication), see the work of Adrian Sampson on approximate computing
https://www.cs.cornell.edu/~asampson/research.html
In particular, the EnerJ paper (PLDI 2011) and Probabilistic Assertions (PLDI 2014).
Update: I maintain my scores after the rebuttal discussion.