How can I exclude certain ports from the firewall notifications? First of all, I know I can manually configure the firewall through IPFW but I hope that the OS X firewall system has an option for this by default.

The problem I am currently having is that when I'm doing web development with Django I get a 'Do you want the application "python" to accept incoming netowrk connections?' every time I save a file within the project.

Note that I need Django to be available throughout the local network since I usually use multiple computers during development (i.e. IE box to test) so I cannot set it to localhost only.

Is there any way to have OS X remember this "Allow" permanently instead of asking every time I save a file?
The dialog is not coming from ipfw but from socketfilterfw, the OS X Application based firewall. That one is not port based, like ipfw but based on processes. If an application/script wants to bind a port the application firewall checks it's ruleset and subsequently asks the user if that should be allowed which is what you already see. If you allow this, the binary in question gets signed and is allowed access in the future without user interaction. If the binary changes, you get asked again.  

The only way to prevent getting asked is to turn off the app firewall which I do not recommend. Alternatively you could start signing each change of your python files which is likely to become annoying.

Sidenote: If you need a port based firewall as well you should be using pf (pf.conf(5), pfctl(8)) on Mountain-Lion as ipfw is deprecated.