<p>I am faced with a problem which I bet was already solved before, but that I had never seen. Perhaps by discussing it abstractly, someone can point me to relevant literature.</p>
<p>It goes like this: I have a dataset of images <span class="math-container">$I_j$</span> and numerical features <span class="math-container">$f_{1,j}, f_{2,j}, ..., f_{k, j}$</span>. In production, I don't have access to the features <span class="math-container">$f_{i,j}$</span>, which I know to be more relevant to my learning problem than the images.</p>
<p>However, theses features <span class="math-container">$f_{i,j}$</span> are also very hard to obtain, and there is a known correlation (informal) between the images and the numerical features <span class="math-container">$f_{i,j}$</span>. So the whole point of using the images is to avoid using the <span class="math-container">$f's$</span>.</p>
<p>Know comes the question: is there any useful way I could learn a representation of the images that &quot;exposes&quot; the maximum information about the features <span class="math-container">$f_{i,j}$</span>? Perhaps some form of autoencoder with a regularization term that penalizes uncorrelated codings?</p>
<p>Any ideas are welcome. Thanks!</p>
<p>Edit:</p>
<p>In the problem that I have, the images <span class="math-container">$I_j$</span> are <span class="math-container">$(64,64)$</span> images of tumor nuclei, and the <span class="math-container">$f_{i,j}$</span> are genetical information about each cell, such as the expressivity of certain biomarks (HER2, ER, etc.).</p>
<p><a href="https://i.sstatic.net/RmHCc.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/RmHCc.png" alt="enter image description here" /></a></p>

CC BY-SA 4.0
2023-08-16T12:12:01.623Z
can you please provide a practical example of what $f_{i,j}$ is?
Alberto
CC BY-SA 4.0
2023-08-16T12:39:26.233Z
@AlbertoSinigaglia Done.
Alek Fröhlich
CC BY-SA 4.0
2023-08-16T13:19:24.080Z
It looks like the features you are wanting to learn are explanatory for some other classification that you don't say? And that you want to train a model that predicts the classification from an image as the primary task, whilst also having these features expressed within the model? Could you please clarify
Neil Slater
CC BY-SA 4.0
2023-08-16T13:53:35.883Z
@NeilSlater A first example could be to classify a cell as HER2+ or HER2- based only on the image data. (During training the autoencoder would have access to reliable HER2 score for each nuclei)
Alek Fröhlich
CC BY-SA 4.0
2023-08-16T23:09:10.177Z
I'm not really an expert on this field so i still don't get what are the $f$, but I think that you can formulize this as a contrastive problem, using biomarks to attract of pull away latent vectors
Alberto
<regularization><representation-learning><correlation>
Does this learning scenario have a name? If so, can someone point me to relevant literature?
Alek Fröhlich
# Does this learning scenario have a name? If so, can someone point me to relevant literature?

(Asked by: username_0 on 2023-08-16)

I am faced with a problem which I bet was already solved before, but that I had never seen. Perhaps by discussing it abstractly, someone can point me to relevant literature.

It goes like this: I have a dataset of images $I\_j$ and numerical features $f\_{1,j}, f\_{2,j}, ..., f\_{k, j}$. In production, I don't have access to the features $f\_{i,j}$, which I know to be more relevant to my learning problem than the images.

However, theses features $f\_{i,j}$ are also very hard to obtain, and there is a known correlation (informal) between the images and the numerical features $f\_{i,j}$. So the whole point of using the images is to avoid using the $f's$.

Know comes the question: is there any useful way I could learn a representation of the images that "exposes" the maximum information about the features $f\_{i,j}$? Perhaps some form of autoencoder with a regularization term that penalizes uncorrelated codings?

Any ideas are welcome. Thanks!

Edit:

In the problem that I have, the images $I\_j$ are $(64,64)$ images of tumor nuclei, and the $f\_{i,j}$ are genetical information about each cell, such as the expressivity of certain biomarks (HER2, ER, etc.).

[![enter image description here](https://i.sstatic.net/RmHCc.png)](https://i.sstatic.net/RmHCc.png)

**Tags:** regularization, representation-learning, correlation

> **Comments:**
> **username_1**: can you please provide a practical example of what $f_{i,j}$ is?  
> **username_0**: @username_1Sinigaglia Done.  
> **username_2**: It looks like the features you are wanting to learn are explanatory for some other classification that you don't say? And that you want to train a model that predicts the classification from an image as the primary task, whilst also having these features expressed within the model? Could you please clarify  
> **username_0**: @NeilSlater A first example could be to classify a cell as HER2+ or HER2- based only on the image data. (During training the autoencoder would have access to reliable HER2 score for each nuclei)  
> **username_1**: I'm not really an expert on this field so i still don't get what are the $f$, but I think that you can formulize this as a contrastive problem, using biomarks to attract of pull away latent vectors  

---

## 0 answers

