<p>What you want to do is called multi-task learning. Here's what you do:</p>
<ol>
<li>Create a second Input.</li>
<li>Attach it to 1D CNN (2-3 layers), so it aggregates this tabular information.</li>
<li>Concatenate this feature with the intermediate feature generated by the U-Net using Concatenate layer.</li>
<li>Put a dense layer of 2 after this.</li>
<li>Put softmax with units = number of classes.</li>
<li>Add CE loss calculated with this one and the ground labels to the loss of U-Net.</li>
</ol>
<p>This is in regards to TensorFlow. The same can be done in PyTorch easily.</p>

CC BY-SA 4.0
2021-03-18T17:29:08.310Z
2021-03-18T17:29:08.310Z
Abhishek Verma
<p>I am using a U-Net to segment cancer cells in images of patients' arms. I would like to add patient data to it in order to see if it is possible to enhance the segmentation (patient data comes in the form of a table containing features such as gender, age, etc.). So far, my researches have led me nowhere. What can I do in order to achieve this?</p>

<deep-learning><data-preprocessing><u-net><semantic-segmentation><multi-task-learning>
How should I incorporate numerical and categorical data as part of the inputs to the U-net for semantic segmentation?
Skyris
# How should I incorporate numerical and categorical data as part of the inputs to the U-net for semantic segmentation?

(Asked by: username_0 on 2021-03-18)

I am using a U-Net to segment cancer cells in images of patients' arms. I would like to add patient data to it in order to see if it is possible to enhance the segmentation (patient data comes in the form of a table containing features such as gender, age, etc.). So far, my researches have led me nowhere. What can I do in order to achieve this?

**Tags:** deep-learning, data-preprocessing, u-net, semantic-segmentation, multi-task-learning


---

## 1 answers

### Answer by username_1 on 2021-03-18. Score: 2 | Accepted answer

What you want to do is called multi-task learning. Here's what you do:

1. Create a second Input.
2. Attach it to 1D CNN (2-3 layers), so it aggregates this tabular information.
3. Concatenate this feature with the intermediate feature generated by the U-Net using Concatenate layer.
4. Put a dense layer of 2 after this.
5. Put softmax with units = number of classes.
6. Add CE loss calculated with this one and the ground labels to the loss of U-Net.

This is in regards to TensorFlow. The same can be done in PyTorch easily.


---

