Thanks for the information!
How do you process the complex input? Are you using an Embedding layer?
The output of the concatenated outputs looks a bit redundant to me.
For example, for model1 and the complex input, these seem to be a lot of repetitions for each sample.