Answers.Body: <p>The general version of your problem is described a bit in <a href="https://stats.stackexchange.com/questions/341740/handling-missing-data-for-a-neural-network">this stackexchange post</a>.
  However, it's hard to know how to address the issue unless the missing file dropout is better understood. For example, if the missing files are truly iid, then zeroing out the tensor might be ok. If one of your files drops out 2 or 3x more than another, zeroing out will skew towards the other features.</p>
  <p>You also specified that the amount of data you have is small, which might make dropout less effective with a net, and you might have better luck with hand crafted features with a decision tree.</p>
  <p>Ultimately, you just have to try things. I would start by making sure you can overfit on the small set of examples without missing data and then try to overfit again with the missing data to see how, or if, your metrics degrade. This will give you better insight for what to try when you train with a proper train / test / validation split.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-28T12:57:04.837Z
Answers.LastActivityDate: 2022-12-28T12:57:04.837Z
Answers.UserDisplayName: juicedatom
Answers.Body: <p>I am not sure if I understood your question. Basically, you have data with different dimensions. For example you have image data HxWxc (3dimension) and you have video data, HxWxcxt (4dimension). What you are saying is that padding zeros to the image to make a video. That does not really make sense to me.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-30T16:34:21.593Z
Answers.Text: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-30T18:42:16.123Z
Answers.Text: No, I mean that some files are missing altogether.
Answers.UserDisplayName: Paul Reiners
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-30T15:41:44.000Z
Answers.LastActivityDate: 2022-07-30T15:41:44.000Z
Answers.UserDisplayName: ai-py
Body: <p>I'm trying to solve a medical imaging regression problem using a CNN.  Each of the samples in my data set consists of one, two, or three of the following file types:</p>
  <ul>
  <li>flair.nii.gz</li>
  <li>mprage.nii.gz</li>
  <li>swi.nii.gz</li>
  </ul>
  <p>Each of the files is a three- or four-dimensional matrix of voxel values between 0 and 255.</p>
  <p>I don't want to throw information away since I have a limited number of data samples.  Is there a technique for working around those cases which have a 'missing' file (or two)?  For example, could I construct a matrix consisting of, say, all zeroes of the correct dimension and size and use this as a replacement for the file?  Would this work or would it lead to problems with the CNN?  Of course, the samples that I want to make inferences on might also have missing files, so, maybe, a matrix with all zeroes would be a type of information.</p>
  <p>Anyway, what does one do in a case like this?</p>
  
Tags: <deep-learning><convolutional-neural-networks><regression>
Title: Dealing with incomplete file sets for a CNN for medical imaging regression problem
UserDisplayName: Paul Reiners
ThreadText: # Dealing with incomplete file sets for a CNN for medical imaging regression problem
  
  (Asked by: username_0 on 2022-07-29)
  
  I'm trying to solve a medical imaging regression problem using a CNN. Each of the samples in my data set consists of one, two, or three of the following file types:
  
  * flair.nii.gz
  * mprage.nii.gz
  * swi.nii.gz
  
  Each of the files is a three- or four-dimensional matrix of voxel values between 0 and 255.
  
  I don't want to throw information away since I have a limited number of data samples. Is there a technique for working around those cases which have a 'missing' file (or two)? For example, could I construct a matrix consisting of, say, all zeroes of the correct dimension and size and use this as a replacement for the file? Would this work or would it lead to problems with the CNN? Of course, the samples that I want to make inferences on might also have missing files, so, maybe, a matrix with all zeroes would be a type of information.
  
  Anyway, what does one do in a case like this?
  
  **Tags:** deep-learning, convolutional-neural-networks, regression
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-12-28. Score: 0
  
  The general version of your problem is described a bit in [this stackexchange post](https://stats.stackexchange.com/questions/341740/handling-missing-data-for-a-neural-network).
  However, it's hard to know how to address the issue unless the missing file dropout is better understood. For example, if the missing files are truly iid, then zeroing out the tensor might be ok. If one of your files drops out 2 or 3x more than another, zeroing out will skew towards the other features.
  
  You also specified that the amount of data you have is small, which might make dropout less effective with a net, and you might have better luck with hand crafted features with a decision tree.
  
  Ultimately, you just have to try things. I would start by making sure you can overfit on the small set of examples without missing data and then try to overfit again with the missing data to see how, or if, your metrics degrade. This will give you better insight for what to try when you train with a proper train / test / validation split.
  
  
  ---
  
  ### Answer by username_2 on 2022-07-30. Score: -1
  
  I am not sure if I understood your question. Basically, you have data with different dimensions. For example you have image data HxWxc (3dimension) and you have video data, HxWxcxt (4dimension). What you are saying is that padding zeros to the image to make a video. That does not really make sense to me.
  
  > **Comments:**
  > **username_3**: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  > **username_0**: No, I mean that some files are missing altogether.  
  
  ---
  
  