Should my python web app use unicode for all strings?
Using Unicode internally is a good way to avoid problems with non-ASCII characters.  Convert at the boundaries of your application (incoming data to unicode, outgoing data to UTF-8 or whatever).  Pylons can do the conversion for you in many cases: e.g. controllers can safely return unicode strings; SQLAlchemy models may declare Unicode columns.
Regarding string literals in your source code: the u prefix is usually not necessary.  You can safely mix str objects containing ASCII with unicode objects.  Just make sure all your string literals are either pure ASCII or are u"unicode".
I see some frameworks like Django using unicode all over the place so it seems like it might be a good idea.
On the other hand, it seems like a big pain to have all these extra 'u's floating around everywhere. 
What will be a problem if I don't do this?
Are there any issues that will come up if I do do this?
I'm using Pylons right now as my framework.