Answers.Body: <p>As you mentioned in the comments about a possible problem of using mean, median type of imputations naively could lead to wrong predictions. In such cases, you need to first check whether you have <em>enough</em> data. </p>
  
  <h2>If you have enough data</h2>
  
  <p>You can try using <a href="https://www.rdocumentation.org/packages/mice/versions/3.8.0/topics/mice" rel="nofollow noreferrer">MICE</a> (Multivariate Imputation By Chained Equations) algorithm on your missing data. The method is based on Fully Conditional Specification, where each incomplete variable is imputed by a separate model. The MICE algorithm can impute mixes of continuous, binary, unordered categorical and ordered categorical data. One note of caution with this method: It is a computationally expensive method, so it use it if you are not short on time. </p>
  
  <p>The important thing to keep in mind is that in order to tackle such problems, you would be needing multiple iterations as a part of your algorithm. The conventional setup you are describing, does not seem to iterative in nature and hence you are running into the problem of features being input and output at the same time.</p>
  
  <p>If by any chance you insist on finding the missing values just for solving a downstream task like classification or regression, you can try the XgBoost algorithm. It can be used as a classifier or as a regressor. This algorithm can handle missing values inherently. Source: <a href="https://stats.stackexchange.com/questions/235489/xgboost-can-handle-missing-data-in-the-forecasting-phase">this answer</a></p>
  
  <h2>If you don't have enough data</h2>
  
  <p>In such a case you would need to introduce bias in your model using your insights or domain knowledge about the problem. For, e.g., in the a possible problem of estimating weights using heights, you had an insight that your data comprises more of short people. So instead of naively using median values of the total dataset for weight, you can try to bin the data according to their heights, say 'S', 'M', 'L', 'XL', and estimate the weights of each bin separately using the median values of their respective bins. The thing to keep in mind is that when data is low, you need to provide knowledge to the model by enforcing some bias using your insights and domain knowledge about the problem.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-21T11:38:09.290Z
Answers.LastActivityDate: 2020-05-21T11:44:40.437Z
Answers.LastEditDate: 2020-05-21T11:44:40.437Z
Answers.UserDisplayName: user1825567
Body: <p>I have a dataset with missing values, I would like to use machine learning methods to fill.
  In more detail, there are <span class="math-container">$n$</span>
  individuals, for which up to 10 properties are provided, all numerical. The fact is, there are no individuals for which all properties are given. The first rows (each row contains data for a given individual) do look as the following </p>
  
  <p><span class="math-container">\begin{bmatrix}
      1 &amp; NA &amp; 3.6 &amp; 12.1 &amp; NA \\
  1.2 &amp; NA &amp; NA &amp; 4 &amp; NA \\
      NA &amp; 4 &amp; 5 &amp; NA &amp; 7
    \end{bmatrix}</span></p>
  
  <p>What methods could be applicable in general? </p>
  
  <p>I have some basic experience in classifiers and Random Forests. Modulo the obvious difference that this is not a classifying problem, what I struggle most with is that the same variable (described in the e.g 
  <span class="math-container">$n$</span>-th column) is both an input and an output. Say I want to predict the value <span class="math-container">$A_{2,3}$</span> in the dataset above. In this case, all the values in the third column could be used as input, excluded of course <span class="math-container">$A_{2,3}$</span>
  itself, which would be an output.</p>
  
  <p>This seems to be different than the more conventional set-up of predicting a property, given a set of other properties (e.g, predict income given education, work sector, seniority, etc.). In this case, sometimes the income is to be predicted, sometimes used for predicting another variable.
  I am aware of methods which, given a vector <span class="math-container">$X_i$</span>, could approximate a function <span class="math-container">$F$</span> and predict responses <span class="math-container">$Y_i$</span> with</p>
  
  <p><span class="math-container">$$ Y_i = F(X_i)$$</span></p>
  
  <p>In the scenario I described though, it looks like some implicit function <span class="math-container">$\Phi$</span> is to be found, a function of all the variables <span class="math-container">$Z_i$</span> (columns in the dataset above)</p>
  
  <p><span class="math-container">$$ \Phi (Z_i) = 0$$</span></p>
  
  <p>What methods could handle this aspect? I understand the question is probably too general, but I could not find much and could do with a starting point. I would be already content with some hints for my further reading, but anything more would be gratefully welcomed, thanks.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-15T10:24:34.570Z
Comments.Text: Hi and welcome to AI SE! Have you tried the typical methods of filling the missing values with the average (of the non-missing values in the corresponding column/property), zero, minimum, and maximum? I don't know if this will work, but that should be a starting point.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-15T10:43:13.740Z
Comments.Text: Yes that is what I have done so far, but it seems really inaccurate, as it grossly misses some obvious correlations. For example, if an individual is very tall, their weight will also tend to be larger. But if my dataset is populated mainly by short individuals, the average will be skewed and tall individuals will be assigned a smaller weight.
Comments.UserDisplayName: Smerdjakov
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-17T11:11:28.313Z
Comments.Text: Hi, could you explain the data set more, what it represents?
Comments.UserDisplayName: Igor 
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-18T18:27:50.693Z
Comments.Text: @Igor, frankly the question is very general, I do not have a specific dataset in mind, more a general problem, which I thought must be quite common in Data Science. I re-read my post and the beginning might be confusing indeed..
Comments.UserDisplayName: Smerdjakov
Tags: <machine-learning><datasets><data-preprocessing>
Title: How to fill missing values in a dataset where some properties can be inputs and outputs?
UserDisplayName: Smerdjakov
ThreadText: # How to fill missing values in a dataset where some properties can be inputs and outputs?
  
  (Asked by: username_0 on 2020-05-14)
  
  I have a dataset with missing values, I would like to use machine learning methods to fill.
  In more detail, there are $n$
  individuals, for which up to 10 properties are provided, all numerical. The fact is, there are no individuals for which all properties are given. The first rows (each row contains data for a given individual) do look as the following
  
  \begin{bmatrix}
  1 & NA & 3.6 & 12.1 & NA \\
  1.2 & NA & NA & 4 & NA \\
  NA & 4 & 5 & NA & 7
  \end{bmatrix}
  
  What methods could be applicable in general?
  
  I have some basic experience in classifiers and Random Forests. Modulo the obvious difference that this is not a classifying problem, what I struggle most with is that the same variable (described in the e.g
  $n$-th column) is both an input and an output. Say I want to predict the value $A\_{2,3}$ in the dataset above. In this case, all the values in the third column could be used as input, excluded of course $A\_{2,3}$
  itself, which would be an output.
  
  This seems to be different than the more conventional set-up of predicting a property, given a set of other properties (e.g, predict income given education, work sector, seniority, etc.). In this case, sometimes the income is to be predicted, sometimes used for predicting another variable.
  I am aware of methods which, given a vector $X\_i$, could approximate a function $F$ and predict responses $Y\_i$ with
  
  $$ Y\_i = F(X\_i)$$
  
  In the scenario I described though, it looks like some implicit function $\Phi$ is to be found, a function of all the variables $Z\_i$ (columns in the dataset above)
  
  $$ \Phi (Z\_i) = 0$$
  
  What methods could handle this aspect? I understand the question is probably too general, but I could not find much and could do with a starting point. I would be already content with some hints for my further reading, but anything more would be gratefully welcomed, thanks.
  
  **Tags:** machine-learning, datasets, data-preprocessing
  
  > **Comments:**
  > **username_1**: Hi and welcome to AI SE! Have you tried the typical methods of filling the missing values with the average (of the non-missing values in the corresponding column/property), zero, minimum, and maximum? I don't know if this will work, but that should be a starting point.  
  > **username_0**: Yes that is what I have done so far, but it seems really inaccurate, as it grossly misses some obvious correlations. For example, if an individual is very tall, their weight will also tend to be larger. But if my dataset is populated mainly by short individuals, the average will be skewed and tall individuals will be assigned a smaller weight.  
  > **username_2**: Hi, could you explain the data set more, what it represents?  
  > **username_0**: @username_2, frankly the question is very general, I do not have a specific dataset in mind, more a general problem, which I thought must be quite common in Data Science. I re-read my post and the beginning might be confusing indeed..  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2020-05-21. Score: 1 | Accepted answer
  
  As you mentioned in the comments about a possible problem of using mean, median type of imputations naively could lead to wrong predictions. In such cases, you need to first check whether you have *enough* data.
  
  ## If you have enough data
  
  You can try using [MICE](https://www.rdocumentation.org/packages/mice/versions/3.8.0/topics/mice) (Multivariate Imputation By Chained Equations) algorithm on your missing data. The method is based on Fully Conditional Specification, where each incomplete variable is imputed by a separate model. The MICE algorithm can impute mixes of continuous, binary, unordered categorical and ordered categorical data. One note of caution with this method: It is a computationally expensive method, so it use it if you are not short on time.
  
  The important thing to keep in mind is that in order to tackle such problems, you would be needing multiple iterations as a part of your algorithm. The conventional setup you are describing, does not seem to iterative in nature and hence you are running into the problem of features being input and output at the same time.
  
  If by any chance you insist on finding the missing values just for solving a downstream task like classification or regression, you can try the XgBoost algorithm. It can be used as a classifier or as a regressor. This algorithm can handle missing values inherently. Source: [this answer](https://stats.stackexchange.com/questions/235489/xgboost-can-handle-missing-data-in-the-forecasting-phase)
  
  ## If you don't have enough data
  
  In such a case you would need to introduce bias in your model using your insights or domain knowledge about the problem. For, e.g., in the a possible problem of estimating weights using heights, you had an insight that your data comprises more of short people. So instead of naively using median values of the total dataset for weight, you can try to bin the data according to their heights, say 'S', 'M', 'L', 'XL', and estimate the weights of each bin separately using the median values of their respective bins. The thing to keep in mind is that when data is low, you need to provide knowledge to the model by enforcing some bias using your insights and domain knowledge about the problem.
  
  
  ---
  
  