Can I delete missing data? I have a dataset (4,898 X 17,000) that follows 4898 mothers, fathers, and their children over a period of 15 years. The interviews have been conducted at baseline (when the child was born), year-1, year-3, year-5, year-9, and year-15. I want to predict GPA (at year-15) using random forest from a set of features (Gender, household income, household education, Parental involvement in studies, parental expectation, family structure, cognitive and non-cognitive variables. All of these variables except cognitive variables have been captured at year-15. Cognitive variables have been captured at year-9. Year-9 has about 1300 values missing whereas year-15 has about 1454 values missing due to non-response in year-9 and 15. I am quite new to imputation and I am not sure how to use multiple imputations here (especially when there are more than 1000 rows that have missing values for all the columns). Any help in this regard would be very helpful.

It is important to think about the mechanism leading to missing data. There are three kind of missing data that can happen:

*

*Missing completely at random (MCAR). It means that the probability that an entry is missing is the fixed, independent of its (unobserved) value and independent of other variables. In that case, deleting incomplete data is OK and will not bias your result. However, doing multiple imputation may be more efficient, since you don't need to delete any valuable data. It will also depend on how much of your dataset is missing (maybe you will lose too much data doing complete case analysis, or maybe there's so few missing data that it's not worth the effort of imputation).

*Missing at random (MAR). It means that the probability of an entry being missing depends on the other variables, but not on the unobserved value. In this case, ignoring missing data may bias your results, and multiple imputation is recommended.

*Missing not at random (MNAR). In this case, the probability of missingness does depend on the unobserved value. An extreme example of this would be censoring. In this situation, neither imputation nor complete case analysis will remove bias, and there is no general solution here.

If you are sure that you are in a MCAR scenario (unlikely) or that the fraction of missing data is tiny, you can do complete case analysis. Otherwise, you should try imputation. If you are in a MNAR situation, you may have to rethink if your dataset can answer the questions you ask in an unbiased way.
I think multiple imputation may still work for completely missing rows (at least Bayesian model-based imputation would work, I am not sure about other methods), but these rows are not informative at all, so I think it's safe to delete those anyway.
