Answers.Body: <p>In my opinion, it is not because ensemble methods are not good, just the state-of-the-art and Kaggle competitions are two different fields.</p>
  <p>Kaggle competitions can be understood as an industry project where the target (accuracy, distance value, etc) is the most important, and they can select some <strong>computationally expensive</strong> way such as ensemble methods to reach it.</p>
  <p>The state-of-the-art models in other ways belong to the research area, where the most important is the <strong>contribution</strong> for science, you can not just combine a lot of models then call it is the research (and so unfair with some small researcher groups). If you want to contribute something depend on <strong>ensemble idea</strong>, it should be like <a href="https://arxiv.org/abs/1503.02531" rel="nofollow noreferrer">this paper</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-24T15:26:02.743Z
Answers.Text: I think it's possible to publish something that only shows that the combination of multiple models can achieve better performance than single models (maybe in some applied ML journal or something), although no example comes to my mind right now. In any case, you're right that research is different than Kaggle competitions. In research, ideas and novelty are fundamental, while the goal in Kaggle competitions is to achieve the highest performance (according to some metric).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-24T15:31:16.033Z
Answers.Text: Note, though, a lot of research (especially in DL) that attracts people and funds is the one that shows a model is better than another according to some performance measure (for example, AlphaFold). Moreover, novelty doesn't necessarily need to be radical, but it can just be a combination or improvement of other ideas: this is actually very common.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-25T14:45:30.077Z
Answers.Text: @nbro Agree, ResNext in image classification or YOLO in object detection is a good example of ideas combination
Answers.UserDisplayName: CuCaRot
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-24T14:18:47.780Z
Answers.LastActivityDate: 2020-12-25T09:52:17.237Z
Answers.LastEditDate: 2020-12-25T09:52:17.237Z
Answers.UserDisplayName: CuCaRot
Body: <p>What ensemble methods are used in the state-of-the-art models?</p>
  <p>When I surveyed the state-of-the-art methods of classification and detection, e.g. ImageNet, COCO, etc., I noticed that are few or even no references to the use of ensemble methods like bagging or boosting.</p>
  <p>Is it a bad idea to use them?</p>
  <p>However, I observed that many use ensemble in Kaggle competitions.</p>
  <p>What makes it so different between the two groups of researchers?</p>
  
Tags: <machine-learning><object-detection><ensemble-learning><boosting>
Title: What ensemble methods are used in the state-of-the-art models?
UserDisplayName: Terence Hsu
ThreadText: # What ensemble methods are used in the state-of-the-art models?
  
  (Asked by: username_0 on 2020-12-24)
  
  What ensemble methods are used in the state-of-the-art models?
  
  When I surveyed the state-of-the-art methods of classification and detection, e.g. ImageNet, COCO, etc., I noticed that are few or even no references to the use of ensemble methods like bagging or boosting.
  
  Is it a bad idea to use them?
  
  However, I observed that many use ensemble in Kaggle competitions.
  
  What makes it so different between the two groups of researchers?
  
  **Tags:** machine-learning, object-detection, ensemble-learning, boosting
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-12-24. Score: 2 | Accepted answer
  
  In my opinion, it is not because ensemble methods are not good, just the state-of-the-art and Kaggle competitions are two different fields.
  
  Kaggle competitions can be understood as an industry project where the target (accuracy, distance value, etc) is the most important, and they can select some **computationally expensive** way such as ensemble methods to reach it.
  
  The state-of-the-art models in other ways belong to the research area, where the most important is the **contribution** for science, you can not just combine a lot of models then call it is the research (and so unfair with some small researcher groups). If you want to contribute something depend on **ensemble idea**, it should be like [this paper](https://arxiv.org/abs/1503.02531).
  
  > **Comments:**
  > **username_2**: I think it's possible to publish something that only shows that the combination of multiple models can achieve better performance than single models (maybe in some applied ML journal or something), although no example comes to my mind right now. In any case, you're right that research is different than Kaggle competitions. In research, ideas and novelty are fundamental, while the goal in Kaggle competitions is to achieve the highest performance (according to some metric).  
  > **username_2**: Note, though, a lot of research (especially in DL) that attracts people and funds is the one that shows a model is better than another according to some performance measure (for example, AlphaFold). Moreover, novelty doesn't necessarily need to be radical, but it can just be a combination or improvement of other ideas: this is actually very common.  
  > **username_1**: @username_2 Agree, ResNext in image classification or YOLO in object detection is a good example of ideas combination  
  
  ---
  
  