Semi-Supervised Learning with Declaratively Specified Entropy Constraints
We propose a technique for declaratively specifying strategies for semi-supervised learning (SSL). SSL methods based on different assumptions perform differently on different tasks, which leads to difficulties applying them in practice. In this paper, we propose to use entropy to unify many types of constraints. Our method can be used to easily specify ensembles of semi-supervised learners, as well as agreement constraints and entropic regularization constraints between these learners, and can be used to model both well-known heuristics such as co-training, and novel domain-specific heuristics. Besides, our model is flexible as to the underlying learning mechanism. Compared to prior frameworks for specifying SSL techniques, our technique achieves consistent improvements on a suite of well-studied SSL benchmarks, and obtains a new state-of-the-art result on a difficult relation extraction task.
This paper proposes a method to combine (or ensemble) several SSL heuristics (regularizers) by using a Bayesian optimization approach. The basic idea of the proposed method borrowed from the previous method called D-Learner, which is declared in this paper. Therefore, the proposed method is basically a modification or extension of D-Learner, which seems not to be totally novel. In this perspective, this paper is rather incremental than innovative.  The experimental results look fairly well comparing with the methods in previous studies including the baseline D-Learner on the tasks of text classification and relation extraction examined in this paper.  The followings are my main concerns of this paper. If I misunderstand something, please provide rebuttals to my concerns. I am willing to change my score if I think the authors’ rebuttal is reasonable.   1, The proposed method seems to deeply depend on the TensorLog framework. From this perspective, I somehow feel that this paper is rather a technical report than a scientific paper. I would like authors to clearly explain that what the central theory of the proposed method is, or what the unique idea of the proposed method is. For example, the idea of incorporating many SSL regularizers together in a single leaner is an idea of the previous study.   2, For example, the DCE(Supervised) in Table 2(b), which is the baseline of the proposed method, already outperformed the performance for all the methods in Table 2(a). I wonder why the baseline of the proposed method got such higher performance. From this point, I concern the experimental settings used in this paper whether the comparison was performed in a fair condition.   3, The results for DCE (Ours) shown in Tables 1 (a) and (b) differs. I cannot figure out why this difference was induced from. Please clearly explain this.  After reading author's feedback: Thank you for answering my concerns/questions. Unfortunately, I am still not totally convinced the novelty of the proposed method. Moreover, the reason of the supervised DCE-Learner has already outperformed the most of previous methods is still not super clear for me. Is there any particular reason why any of previous methods did not use SL DCE-Learner (or similar strong baseline SL methods)? It is a bit hard for me to "strongly" recommend this paper for the acceptance. In addition, I would recommend authors to report the results of both splits to prevent the misunderstanding in the next version.