Neglected machine learning ideas
One often neglected idea is transduction. The idea is that you can learn to label data better by augmenting your labeled dataset with unlabeled data.The unlabeled data helps discover the structure of the data which in turns helps the supervised learning.There is a lot of 2-phase approach to this problem, by first discovering feature with unsupervised learning and then using those features in supervised learning, but in many cases it's possible to do both at the same time.If you have a generative model for your data, you can treat the labels as potentially missing values and learn the joint distribution.