Hi @ptrblck for your feedback …

If I understand the use case correctly, you would like to use the pretrained embeddings to create the continuous output features for categorical inputs and train a new classifier with it?

YES

If so, then this would be supported and you could either store the output activations for all inputs directly using forward hooks or just use the  nn.Embedding  layer in isolation, feed the input to it, transform the output to numpy arrays, and train your new classifier with it.

I don’t undertand very well … sorry
This the context

  
      github.com
  
  
    jonmunm/nbs/blob/master/Kaggle/UCI Rep - Adults/UCI Rep - Adults.ipynb 2
{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append('../../')\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from datetime import datetime, timedelta\n",
    "\n",
    "from sklearn import model_selection\n",
    "from sklearn.manifold import TSNE\n",
    "from sklearn.metrics import roc_auc_score, accuracy_score\n",
    "from sklearn.preprocessing import FunctionTransformer as FT\n",
    "\n",


  This file has been truncated. show original

  
  
    
    
  
  


I trained a model using a fully conected NN with embeddings. The target es the column income. The model performs well (you can see the metrics).
But now, I want to use XGBoost and give it a try. However, considering the originals dataset’s, income is predicted using the categorical and continius features. So, I don’t want to use OneHot o similars, because I would be losing the semantic between the categorical features, and that’s why I want to feed the embeddings vectors into XGBoost, with the others continius features obviously. So, my questions are:

Can I just take the weights from my nn.Embedding layers from my model UciAdultsClassifier  and feed them into XGBoost? As I said, with also the others continius features. I’m just replacing the form of the categorical features
Do I need to train a separate fully connected NN with just the categorical variables using nn.Embedding layers and predict income? is this correct or the right way of using embeddings vectors in others models than NN’s? Is this approach correct, that is, train a new NN with just nn.Embedding layers to predict income (and not consider the continius features. Danger !!), and then use those embedding vectors or weights to use them in XGBoost?

I hope I’ve explained myself well
Thanks very much, best regards