Hi,
There are a few things you could do here. You basically have semi-supervised learning problem, as for any group you have some data that has a label from that group and some that doesn’t.
As an initial test I would recommend creating a dataset for which each image always has a label from each category. For instance you could take ImageNet and train a network to jointly predict both the object category and the dominant colour in the image. This removes the semi-supervised component and ensures your network can generalise to multiple tasks.
Alternatively every group could have an additional ‘unknown’ label. I’ve never used this approach in practice, so while it’s simple to add on, I don’t know how it would perform.
There has been lots of work on semi-supervised learning, for instance using good old fashioned adversarial nets.
You should be careful to distinguish between the cases where you have:

The labels [blue, red] and [car, bike] and you show it a <blue, truck> instance --> what is appropriate here?
The labels [blue, red] and [car, bike, truck] but your particular instance is a blue truck that is only labelled blue. This is the semi supervised case as the network should induce, from other labelled trucks, that while this is blue it is also a truck.

In case (1) you can really only structure it so that you either have a guaranteed label from every group, or you let the network choose a possible ‘unknown’ option for every group.
Sorry if I’ve only repeated your question and not added any value. This is an interesting problem!

Jordan