The decision is made on a per-operation basis. When we have a choice of multiple backends for a given op, we filter out these that don’t support the options you chose and pick the fastest one.
For example cuDNN 5 doesn’t support dilated convolutions. If your model has both dilated and regular convolutions, THCUNN will be used for dilated and cuDNN for regular ones.
This has the upside of always running at the max speed, across many different systems, that have only a subset of backends (e.g. no CUDA support), based on a single model definition.