Is it possible to run two versions of Python side-by-side?
Absolutely.
If you're on *nix, you'd usually just use make altinstall instead of make install, that way the "python" binary won't get installed/overwritten, but instead you'd have e.g. python2.5 or python2.6 installed. Using a separate --prefix with the configure script is also an option, of course.
Some Linux distributions will have multiple versions available via their package managers. They'll similarly be installed as python2.5 etc. (With the distribution's blessed/native version also installed as the regular python binary.)
Windows users generally just install to different directories.
I've been learning Python for a couple of weeks, and although I've been successfully develop apps for Google App Engine with Python 2.6.5, it specifically requires Python 2.5.
Being mindful of compatibility issues when uploading apps (it's a situation I'd rather avoid while learning Python), I wonder if it's possible to have 2.5 and 2.6.5 installed on the same machine.  Ideally I'd like to use 2.6.5 as the default, and configure GAE to somehow use 2.5.