Question: I'm working with a start-up, mostly doing system administration and I've come across a some security issues that I'm not really comfortable with. I want to judge whether my expectations are accurate, so I'm looking for some insight into what others have done in this situation, and what risks/problems came up. In particular, how critical are measures like placing admin tools behind a vpn, regular security updates (OS and tools), etc. 

Keep in mind that as this is a start-up, the main goal is to get as many features as possible out the door quickly, so I'll need as much justification as I can get to get the resources for security (i.e. downtime for upgrades, dev time for application security fixes).

Background Info:

* Application is LAMP as well as a custom java client-server.
* Over the next 3 months, I project about 10k anonymous visitors to the site and up to 1000 authenticated users.
* Younger audience (16-25) which is guaranteed to have an above average number of black-hats included.

Thanks in advance for your responses, and I'll welcome any related advice.

Answer: 
Make sure you know what version and patch level your servers are running, not just the OS, but all related components and everything that is actually executing the the machine. 
Then make sure you are never more than a day behind.
Not doing so leads to much pain, and you don't hear of most of it - most of my past employers would never publicly admit being hacked as it reflects badly on them, so you can assume systems are getting hacked left and right with pretty serious consequences to companies, you just don't hear about most of these events.
Have a look at Mod Security for the various possibilities in the software setup:
Do a Google search for "mod\_security howto example"

Simple example to start: <http://www.ghacks.net/2009/07/15/install-mod_security-for-better-apache-security/>