OPPORTUNISTIC LEARNING: BUDGETED COST-SENSITIVE LEARNING FROM DATA STREAMS
In many real-world learning scenarios, features are only acquirable at a cost constrained under a budget. In this paper, we propose a novel approach for costsensitive feature acquisition at the prediction-time. The suggested method acquires features incrementally based on a context-aware feature-value function. We formulate the problem in the reinforcement learning paradigm, and introduce a reward function based on the utility of each feature. Specifically, MC dropout sampling is used to measure expected variations of the model uncertainty which is used as a feature-value function. Furthermore, we suggest sharing representations between the class predictor and value function estimator networks. The suggested approach is completely online and is readily applicable to stream learning setups. The solution is evaluated on three different datasets including the well-known MNIST dataset as a benchmark as well as two cost-sensitive datasets: Yahoo Learning to Rank and a dataset in the medical domain for diabetes classification. According to the results, the proposed method is able to efficiently acquire features and make accurate predictions.
This paper presents a novel method for budgeted cost sensitive learning from Data Streams.
This paper seems very similar to the work of Contrado’s RADIN algorithm which similarly evaluates sequential datapoints with a recurrent neural network by adaptively “purchasing” the most valuable features for the current datapoint under evaluation according to a budget. 
In this process, a sample (S_i) with up to “d” features arrives for evaluation.  A partially revealed feature vector x_i arrives at time “t” for consideration.  There seems to exist a set of “known features” that that are revealed “for free” before the budget is considered (Algorithm 1).  Then while either the budget is not exhausted or some other stopping condition is met features are sequentially revealed either randomly (an explore option with a decaying rate of probability) or according to their cost sensitive utility.  When the stopping condition is reached, a prediction is made.  After a prediction is made, a random mini-batch of the partially revealed features is pushed into replay memory along with the correct class label and the P. Q, and target Q networks are updated.
The ideas of using a sequentially revealed vector of features and sequentially training a network are in Contrado’s RADIN paper.   The main novelty of the paper seems to be the use of MC dropout as an estimate of certainty in place of the softmax output layer and the methods of updating the P and Q networks.
The value of this paper is in the idea that we can learn online and in a cost sensitive way.  The most compelling example of this is the idea that a patient shows up at time “t” and we would like to make a prediction of disease in a cost sensitive way.  To this end I would have liked to have seen a chart on how well this algorithm performs across time/history.  How well does the algorithm perform on the first 100 patients vs the last 91,962-91,062 patients at what point would it make sense to start to use the algorithm (how much history is needed).
Am I correct in assuming there are some base features that are revealed “for free” for all samples?  If so how are these chosen?  If so how does the number of these impact the results?  
In Contrado’s RADIN paper the authors explore both the MNIST dataset and others, including a medical dataset “cardio.”  Why did you only use RADIN as a comparison for the MNIST dataset and not the LTRC or diabetes dataset?  Did you actually re-implement RADIN or just take the numbers from their paper?  In which case, are you certain which MNIST set was used in this paper? (it was not as well specified as in your paper).
With respect to the real world validity of the paper, given that the primary value of the paper has to do with cost sensitive online learning, it would have been better to talk more about the various cost structure and how those impact the value of your algorithm.  For the first example, MNIST, the assumed uniform cost structure is a toy example that equates feature acquisition with cost.  The second example uses computational cost vs relevance gain.  This would just me a measure of computational efficiency, in which case all of the computational cost of running the updates to your networks should also be considered as cost.  With respect to the third proprietary diabetes dataset, the costs are real and relevant, however there discussion of these are given except to say that you had a single person familiar with medical billing create them for you (also the web address you cite is a general address and does not go to the dataset you are using). 
 In reality, these costs would be bundled.  You say you estimate the cost in terms of overall financial burden, patient privacy and patient inconvenience.  Usually if you ask the patient to fill out a survey it has multiple questions, so for the same cost you get all the answers.  Similarly if you do a blood draw and test for multiple factors the cost to the patient and the hospital are paid for the most part upfront.  It is not realistic to say that the cost of asking a patient a questions is 1/20th of the cost of the survey.  The first survey question asked would be more likely 90-95% of the cost with each additional question some incremental percentage.  To show the value of your work, a better discussion of the cost savings would be appreciated.