Question: Every now and then I get the "zlibinstream:exceeded bytesIn" error when trying to connect to my server through my vnc. Why do I get this error and how can I solve it when I get it - without restarting my server ?

Answer: 
Came across the same issue a couple of times. I am not aware of permanent fix, but for a quick one just change colour to Fullcolour.

For me in VNC Viewer it was in options/Expert/FullColour to true.
In my case the solutions is just to change quality to `Low`. Seems to work fine.