Question: Every now and then I get the "zlibinstream:exceeded bytesIn" error when trying to connect to my server through my vnc. Why do I get this error and how can I solve it when I get it - without restarting my server ?

Answer: 
Came across the same issue a couple of times. I am not aware of permanent fix, but for a quick one just change colour to Fullcolour.

For me in VNC Viewer it was in options/Expert/FullColour to true.
**Another quick fix (for Mac):**
================================

Reading the other responses and trying out a few, I feel toggling random Properties/Options will "solve" this problem. Do the following for Real VNC on Mac:

Options -> Picture Quality -> High. Then Set it back to Automatic.