Is it possible to use both CPU and GPU during training to avoid bellow error ?

CUDA error: out of memory

I have CPU: 32G RAM and GPU: 8G RAM.
When I train on smaller network with batch size =4 , it is OK. But I can not increase batch size, because it faces CUDA out of memory. In this case, it uses just 20% of CPU and all GPU capacity.
Is it possible to set data in CPU and model in GPU? How ??
Thanks