After I delete an app on my iPhone, iTunes copies it over on next sync On my iPhone, I go to the App Store. I download some app. It's crap, so I delete it, by clicking the 'x' when the icons start wobbling.

Some time later, I turn on my mac, and iTunes starts syncing with my iPhone. For some reason, iTunes copies the APP I DELETED over to my iPhone, so I have to delete it again. Only when I delete it again will it stop copying it.

I find it hard to believe this is a design decision. It would be understandable to keep the app on my computer, but why does it copy it over to my iPhone even though I deleted it there?

Specs: latest version of iTunes on latest version of Lion, and latest version of iOS on iPhone 4.
The issue here is to do with the Automatic Downloads feature of iTunes and iOS. I believe (not 100% sure) that when you update to iTunes X and iOS 5 the feature to automatically sync apps between devices is enabled by default, and it is up to the user to disable it. 

Basically, when you download an app with this feature enabled, it will download a copy to your computer also. You can then delete this app from the iPhone, but however it will remain on iTunes.

Now, when you plug your iPhone in, there is another setting that means the deleted app will be sent straight from iTunes back onto your iPhone. Navigate to the App section in the iPhone tab when it is connected in iTunes, and beneath the list of apps that will sync there is an option to 'Automatically sync new apps.' This causes iTunes to send that app it copied when you originally downloaded it from your iPhone straight back onto your iPhone. So, to fix your problem, deselect this. 

If you want to disable the automatic app downloads completely, in iTunes go to Preferences - Store - and deselect the 'Apps' checkbox. On the iPhone, navigate to Settings - Store, and do the same.