For the complex input I used CNNs followed by an RNN and finally a few linear layers(like a few other research papers in this field). The output you see here:

model 1 on complex input
Variable containing:
4.7969e-04 4.7969e-04 4.7969e-04 … 4.7969e-04 4.7969e-04 4.7969e-04
4.8490e-04 4.8490e-04 4.8490e-04 … 4.8490e-04 4.8490e-04 4.8490e-04

is after CNN layers, RNN and one FC layer.
I tried using embedding but the results were worse for some reason (it didn’t train or trained really slow). Do you think I should give embedding another chance ?
Now that you’ve mentioned it, it does look redundant, is there something I can learn from this ? (and by the way it’s before the concatenation… I concatenate this output with the one I pasted after it and then pass it to two additional FC layers.
To conclude, I have two models that predict the same thing based on two different features, one model is better for some samples and other model is better with other samples.
So, either I need to find a way to train one big model on all features that won’t “fixate” on one feature or connect those two models. I can connect and hope deep learning will successfully use the accuracy of each model, or manually decide which model to use (I might try training another model simply to decide which model to use for each sample, maybe it will give me some interesting results)
Any additional suggestions or opinions are welcome.