text: Probably referring to the fact that you see each other less and less and eventually it could become once a year; you also could have mis-heard him.  The more you dwell on these kind of things the worse it seems- you should have called him soon after and laughed, saying "See you next year?! Very funny!".  That would have broken the ice- you can still do this if it hasn't been too long...