How to prevent Terminal from resizing when font size is changed 10.8.3/Terminal 2.3.

Terminal resizes the entire window in seemingly unpredictable ways whenever I change the size of the font -- a frequent operation when frequently going between laptop and external displays.

Is there some way to prevent this?  iTerm seems to do the same thing.

Update:  Based on @robmathers' answer below, it seems likely that what I want isn't possible.  Can anyone recommend any OS X terminal emulators that don't have this feature?
Old question here ... but thought I'd add that this has been supported in iTerm for almost two years now https://gitlab.com/gnachman/iterm2/issues/2900

Just uncheck the "Adjust window when changing font size" box in the preferences.



2020 Update

The box is now here: