If I understand the use case correctly, raw_data is some kind of temporal signal, while sex and age are some independent features.
I would try to apply the conv layer on the temporal signal alone, use pass the other features e.g. to a linear layer, and concatenate the outputs afterwards.
Alternatively, you could of course try to add sex and age to the temporal signal, but I’m not sure, if your conv layer will capture the information accordingly.