Sounds strange.
Did you use the volatile=True param on your Variables?
Is the batch size larger during eval than train?
Do you use cuDNN in both cases?
Could you post a code snippet reproducing the issue?