Question: Every now and then I get the "zlibinstream:exceeded bytesIn" error when trying to connect to my server through my vnc. Why do I get this error and how can I solve it when I get it - without restarting my server ?

Answer: 
Came across the same issue a couple of times. I am not aware of permanent fix, but for a quick one just change colour to Fullcolour.

For me in VNC Viewer it was in options/Expert/FullColour to true.
What worked for me was to turn off "`Adapt to network speed (recommended)`" (under `Options->Display`) and then move the compression slider to the left from "`All available colors, medium compression`" to "`Hundreds of colors`".

Prior to this, Burpy Burp's answer of changing options/Expert/FullColour to true temporarily fixed the problem but it came back 22 days later.