The Three Cultures of Machine Learning
The Geneticists: Use evolutionary principles to have a model organize itself<p>The Bayesians: Pick good priors and use Bayesian statistics<p>The Symbolists: Use top-down approaches to modeling cognition, using symbols and hand-crafted features<p>The Conspirators: Hinton, Lecun, Bengio et al. End-to-end deep learning without manual feature engineering<p>The Swiss School: Schmidhuber et al. LSTM&#x27;s as a path to general AI.<p>The Russians: Use Support Vector Machines and its strong theoretical foundation<p>The Competitors: Only care about performance and generalization robustness. Not shy to build extremely slow and complex models.<p>The Speed Freaks: Care about fast convergence, simplicity, online learning, ease of use, scalability.<p>The Tree Huggers: Use mostly tree-based models, like Random Forests and Gradient Boosted Decision Trees<p>The Compressors: View cognition as compression. Compressed sensing, approximate matrix factorization<p>The Kitchen-sinkers: View learning as brute-force computation. Throw lots of feature transforms and random models and kernels at a problem<p>The Reinforcement learners: Look for feedback loops to add to the problem definition. The environment of the model is important.<p>The Complexities: Use methods and approaches from physics, dynamical systems and complexity&#x2F;information theory.<p>The Theorists: Will not use a method, if there is no clear theory to explain it<p>The Pragmatists: Will use an effective method, to show that there needs to be a theory to explain it<p>The Cognitive Scientists: Build machine learning models to better understand (human) cognition<p>The Doom-sayers: ML Practitioners who worry about the singularity and care about beating human performance<p>The Socialists: View machine learning as a possible danger to society. Study algorithmic bias.<p>The Engineers: Worry about implementation, pipe-line jungles, drift, data quality.<p>The Combiners: Try to use the strengths of different approaches, while eliminating their weaknesses.<p>The Pac Learners: Search for the best hypothesis that is both accurate and computationally tractable.<p>See also <a href="http:&#x2F;&#x2F;www.kdnuggets.com&#x2F;2015&#x2F;03&#x2F;all-machine-learning-models-have-flaws.html" rel="nofollow">http:&#x2F;&#x2F;www.kdnuggets.com&#x2F;2015&#x2F;03&#x2F;all-machine-learning-models...</a><p>&gt; It is common for people to learn about machine learning within one framework which often becomes there &quot;home framework&quot; through which they attempt to filter all machine learning. (Have you met people who can only think in terms of kernels? Only via Bayes Law? Only via PAC Learning?) Explicitly understanding the existence of these other frameworks can help resolve the confusion.