Algorithmic Assurance: An Active Approach to Algorithmic Testing using Bayesian Optimisation
We introduce algorithmic assurance, the problem of testing whether machine learning algorithms are conforming to their intended design goal. We address this problem by proposing an efficient framework for algorithmic testing. To provide assurance, we need to efficiently discover scenarios where an algorithm decision deviates maximally from its intended gold standard. We mathematically formulate this task as an optimisation problem of an expensive, black-box function. We use an active learning approach based on Bayesian optimisation to solve this optimisation problem. We extend this framework to algorithms with vector-valued outputs by making appropriate modification in Bayesian optimisation via the EXP3 algorithm. We theoretically analyse our methods for convergence. Using two real-world applications, we demonstrate the efficiency of our methods. The significance of our problem formulation and initial solutions is that it will serve as the foundation in assuring humans about machines making complex decisions.
The paper addresses the question: how can we be sure that an ML system behaves as intended. A straight forward method for this assessment is presented, some fundamental properties of the solution are discussed, and the method is tested exemplarily on two real word tasks.  Addressing the reliability of ML solutions is, in my opinion, a very important topic. I believe that the discussion will gain intensity. This contribution provides one systematic way to address one of the questions, concerning reliability. As the authors claim---the paper presents "an early attempt towards this goal.".   The paper is well written and organized. Still there seem to be a few textual errors, like page 2 line 45-46 "shows that it common to", page 4, line 188 "we use Hedge algorithm", and page 7, line "whose % compositions".  Starting on page 7, line 252 there are blanks before the % sign. The general rule is that no blank is used between a number and the percent sign, as is correctly written on page 8, line 294.  In addition I assume, that for the final version, the missing capitalization in the literature will be fixed, as in ". hedge", or "gaussian", "bayesian".  On page 1, line 13 is written "building trust between humans and machines". This sounds strange to me. I think it is all about the trust, that humans can have in machines, and I do not think that the trust a machine might or might not have in humans is addressed here. Same for page 3, line 95.  There is another question I'd like to ask. I'm not sure if the question or the answer to it is fitting in the text, but I ask anyway: If the process of algorithmic assurance makes more effort and needs more expensive labeling by the expert as the initial training of the ML solution, is this something that cannot be avoided, something that we need to accept if we want to get this assurance? And what about the data gather by the process of algorithmic assurance? We could use this data to improve the initial ML solution. Do we not consider this, because this line of thought will only lead us in a recursion. It might be just like not using a validation set for training. It might also be possible, that one could asses the quality of the initial ML solution and then use the new data to train a second, improved ML solution, and then, without assessing the new ML solution, assume that the error is no bigger than the error of the initial ML solution.   Young topic, many questions.  Post Rebuttal =========== The Author Feedback has sufficiently clarified my questions.