Question: Every now and then I get the "zlibinstream:exceeded bytesIn" error when trying to connect to my server through my vnc. Why do I get this error and how can I solve it when I get it - without restarting my server ?

Answer: 
What worked for me was to turn off "`Adapt to network speed (recommended)`" (under `Options->Display`) and then move the compression slider to the left from "`All available colors, medium compression`" to "`Hundreds of colors`".

Prior to this, Burpy Burp's answer of changing options/Expert/FullColour to true temporarily fixed the problem but it came back 22 days later.
Opening the options dialog without changing anything did it for me. **Baffled**.