Answers.Body: <p>No. Ensemble Learning (EL) is a way to improve the model performance, which usually means to <em>reduce bias</em> (i.e., get a better model class) or <em>reduce variance</em> (i.e., get better at generalizing across samplings of the dataset.)</p>
  <p>Moreover, in EL you need to ensure a &quot;property&quot; (I'll explain in a bit) among the learners (i.e., the models in the ensemble.), which also defines the kind of EL:</p>
  <ul>
  <li>In <strong>boosting</strong> we need the learners to be <em>correlated</em> (this is the property I was referring to) to each other to effectively reduce the bias.</li>
  <li>Conversely, in <strong>bagging</strong> we need the models to do <em>independent</em> errors otherwise we can't reduce variance by averaging them out.</li>
  </ul>
  <p>In your case you're <em>chaining</em> two models, thus not combining/aggregating their predictions. When you chain one or more models you have to consider that each model made an error, say <span class="math-container">$e$</span>, and that each time you chain the next model would receive an input (which is the output of the previous model) that already incurs an error <span class="math-container">$e$</span>, therefore predicting an output associated to an even higher error, say <span class="math-container">$e'$</span> with <span class="math-container">$e'&gt;e$</span>. Usually it happens that the whole error, say <span class="math-container">$E=e+e'+\cdots$</span>, behaves like the sum of individual errors which can even grow quite easily, making your own pipeline drift away quite easily.</p>
  <p>A way to mitigate this issue, is to train the whole pipeline of models in an <em>end-to-end</em> fashion (i.e., you back-propagate the error through all the models), instead of training each model in isolation until the local optimal is reached.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-12T21:06:15.793Z
Answers.LastActivityDate: 2023-12-12T21:06:15.793Z
Answers.UserDisplayName: Luca Anzalone
Answers.Body: <blockquote>
  <p>Would a system like this, which takes the output of one model and uses it as the input for another, be considered Ensemble Learning?</p>
  </blockquote>
  <p>Not usually. The main criteria to consider something as an <a href="https://en.wikipedia.org/wiki/Ensemble_learning" rel="nofollow noreferrer">ensemble learning</a> setup is that there are multiple different learning units -maybe different algorithm - all working with the same inputs and target values. They are then aggregated in some way (taking a mean value, voting, maybe weighted) to get the ensemble's output.</p>
  <p>So to take your example, if you ran multiple types of binary classifier to classify the text at the end of the pipeline, and combined their outputs using a majority vote rule, then that part of your complex model would be considered as an ensemble.</p>
  <p>These terms for system architecture - pipeline, ensemble, stacking etc - are not necessarily formal though. They are for communication about engineering setup, more than mathematical definitions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-12T19:43:53.887Z
Answers.LastActivityDate: 2023-12-13T08:15:11.910Z
Answers.LastEditDate: 2023-12-13T08:15:11.910Z
Answers.UserDisplayName: Neil Slater
Body: <p>For example, if I have a problem in which I try to predict if it is a nice day for jogging from a corpus of images, I might first convert the images to text descriptions (ex. raining in forrest, cloudy in city) and then use a binary classifier to determine if those descriptions are describing a nice day for jogging or not.</p>
  <p>Would a system like this, which takes the output of one model and uses it as the input for another, be considered Ensemble Learning?</p>
  
Tags: <machine-learning><terminology><definitions><ensemble-learning>
Title: Would a pipeline of different models be considered Ensemble Learning?
UserDisplayName: user3517818
ThreadText: # Would a pipeline of different models be considered Ensemble Learning?
  
  (Asked by: username_0 on 2023-12-12)
  
  For example, if I have a problem in which I try to predict if it is a nice day for jogging from a corpus of images, I might first convert the images to text descriptions (ex. raining in forrest, cloudy in city) and then use a binary classifier to determine if those descriptions are describing a nice day for jogging or not.
  
  Would a system like this, which takes the output of one model and uses it as the input for another, be considered Ensemble Learning?
  
  **Tags:** machine-learning, terminology, definitions, ensemble-learning
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-12-12. Score: 3
  
  No. Ensemble Learning (EL) is a way to improve the model performance, which usually means to *reduce bias* (i.e., get a better model class) or *reduce variance* (i.e., get better at generalizing across samplings of the dataset.)
  
  Moreover, in EL you need to ensure a "property" (I'll explain in a bit) among the learners (i.e., the models in the ensemble.), which also defines the kind of EL:
  
  * In **boosting** we need the learners to be *correlated* (this is the property I was referring to) to each other to effectively reduce the bias.
  * Conversely, in **bagging** we need the models to do *independent* errors otherwise we can't reduce variance by averaging them out.
  
  In your case you're *chaining* two models, thus not combining/aggregating their predictions. When you chain one or more models you have to consider that each model made an error, say $e$, and that each time you chain the next model would receive an input (which is the output of the previous model) that already incurs an error $e$, therefore predicting an output associated to an even higher error, say $e'$ with $e'>e$. Usually it happens that the whole error, say $E=e+e'+\cdots$, behaves like the sum of individual errors which can even grow quite easily, making your own pipeline drift away quite easily.
  
  A way to mitigate this issue, is to train the whole pipeline of models in an *end-to-end* fashion (i.e., you back-propagate the error through all the models), instead of training each model in isolation until the local optimal is reached.
  
  
  ---
  
  ### Answer by username_2 on 2023-12-12. Score: 2
  
  > Would a system like this, which takes the output of one model and uses it as the input for another, be considered Ensemble Learning?
  
  Not usually. The main criteria to consider something as an [ensemble learning](https://en.wikipedia.org/wiki/Ensemble_learning) setup is that there are multiple different learning units -maybe different algorithm - all working with the same inputs and target values. They are then aggregated in some way (taking a mean value, voting, maybe weighted) to get the ensemble's output.
  
  So to take your example, if you ran multiple types of binary classifier to classify the text at the end of the pipeline, and combined their outputs using a majority vote rule, then that part of your complex model would be considered as an ensemble.
  
  These terms for system architecture - pipeline, ensemble, stacking etc - are not necessarily formal though. They are for communication about engineering setup, more than mathematical definitions.
  
  
  ---
  
  