If you have that possibility, you can also use the motherboard’s direct output for rendering, which then uses more CPU, but doesn’t require having an additional GPU, if available space is limited for example… As always, it’s a tradeoff…
You can then block Xorg from using the GPU and free a bit of GPU memory this way (I think this was what I followed but can’t remember for sure: https://gist.github.com/alexlee-gk/76a409f62a53883971a18a11af93241b 15).