Answers.Body: <p>The predictions of each of your initial models will become a feature to feed the meta learner. If you use <span class="math-container">$n$</span> initial models, then for each example you will feed the meta learner <span class="math-container">$n$</span> features, each feature being a prediction from one of the initial models. </p>
  
  <p>Note that this doesn’t mean the size of your dataset increases. Instead, each member of <span class="math-container">$X_{val}$</span> will be represented by <span class="math-container">$n$</span> features, with the <span class="math-container">$k$</span>th feature being equal to the initial prediction of the <span class="math-container">$k$</span>th initial predictor.</p>
  
  <p><a href="https://link.medium.com/nHYrgZmXfW" rel="nofollow noreferrer">This medium link</a> recommends a package that can help you build a pipeline. </p>
  
  <p><strong>Note</strong>: in a sense if you are caching the intermediate predictions of each initial predictor, you’ll end up with <span class="math-container">$n$</span> times as many data points as there are entries in <span class="math-container">$X_{val}$</span>. But these are simply each going to be a feature of another data point, so the dataset hasn’t really increased in size. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-28T13:56:17.307Z
Answers.LastActivityDate: 2019-04-28T15:02:15.880Z
Answers.LastEditDate: 2019-04-28T15:02:15.880Z
Answers.UserDisplayName: Hanzy
Body: <p>Suppose we have a data set <span class="math-container">$X$</span> that is split as <span class="math-container">$X_{\text{train}}$</span>, <span class="math-container">$X_{\text{val}}$</span> and <span class="math-container">$X_{\text{test}}$</span> and the outcome variable is binary. Let's say we train three different models (logistic regression, random forest, and a support vector machine) using <span class="math-container">$X_{\text{train}}$</span>. We then get predictions for <span class="math-container">$X_{\text{val}}$</span> using each of the three models. </p>
  
  <p>In <a href="https://qr.ae/TWpjIz" rel="nofollow noreferrer">stacking</a>, is it correct to say that we train a logistic regression model on a data set of dimension <span class="math-container">$|X_{\text{val}}| \times 3$</span> with the predicted values and actual values of the validation set? This logistic regression model is then used to predict outcomes for data in <span class="math-container">$X_{\text{test}}$</span>? </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-28T12:25:12.677Z
Comments.Text: By stacking, do you mean ensemble learning?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-28T13:59:12.787Z
Comments.Text: Do you mind if I edit the title of your question to be a bit more general so it is more clear the context of the post?
Comments.UserDisplayName: Hanzy
Tags: <machine-learning><datasets>
Title: Do we train a logistic regression model using a dataset that is 3 times bigger than the validation dataset?
UserDisplayName: tomoke
ThreadText: # Do we train a logistic regression model using a dataset that is 3 times bigger than the validation dataset?
  
  (Asked by: username_0 on 2019-04-26)
  
  Suppose we have a data set $X$ that is split as $X\_{\text{train}}$, $X\_{\text{val}}$ and $X\_{\text{test}}$ and the outcome variable is binary. Let's say we train three different models (logistic regression, random forest, and a support vector machine) using $X\_{\text{train}}$. We then get predictions for $X\_{\text{val}}$ using each of the three models.
  
  In [stacking](https://qr.ae/TWpjIz), is it correct to say that we train a logistic regression model on a data set of dimension $|X\_{\text{val}}| \times 3$ with the predicted values and actual values of the validation set? This logistic regression model is then used to predict outcomes for data in $X\_{\text{test}}$?
  
  **Tags:** machine-learning, datasets
  > **Comments:**
  > **username_1**: By stacking, do you mean ensemble learning?  
  > **username_2**: Do you mind if I edit the title of your question to be a bit more general so it is more clear the context of the post?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-04-28. Score: 0
  
  The predictions of each of your initial models will become a feature to feed the meta learner. If you use $n$ initial models, then for each example you will feed the meta learner $n$ features, each feature being a prediction from one of the initial models.
  
  Note that this doesn’t mean the size of your dataset increases. Instead, each member of $X\_{val}$ will be represented by $n$ features, with the $k$th feature being equal to the initial prediction of the $k$th initial predictor.
  
  [This medium link](https://link.medium.com/nHYrgZmXfW) recommends a package that can help you build a pipeline.
  
  **Note**: in a sense if you are caching the intermediate predictions of each initial predictor, you’ll end up with $n$ times as many data points as there are entries in $X\_{val}$. But these are simply each going to be a feature of another data point, so the dataset hasn’t really increased in size.
  
  
  ---
  
  