TheOraware:

The improvement i am measuring by feeding the latent/bottleneck into a boosted algo LGBM for classification , i mean after autoencoder training i use the same autoencoder model to get the latent space of my training data and then use this latent space to train classification using LGBM. Before using this above technique as mentioned above two heads , LGBM model accuracy was 0.58 which increase to after using it to 0.59


How was the accuracy of the classifier head in the autoencoder? It would perform a similar task as the LGBM model, i.e. using the latent tensors to classify the samples.



 TheOraware:

I noticed there are data issue as well , data are too fuzzy you cant discriminate the two records which more or less are same but one has target value 1 and other is 0 , all features has the same problem and this problem is widespread in all rows , very very overlapped data


I don’t know how you’ve checked this, but in case both classes are drawn from the same distribution, I would assume that your model should not be able to learn a lot (besides overfitting).