Question: I am starting to read up on SP development but none of the books I have checked have been able to answer this seemingly simple question.

I am pretty new to the dev side of things and feel like I missed something obvious.

Answer: 
A feature is a development package that can be removed and deployed at will from the SharePoint side. Let's say we have a feature in Visual Studios that is a web part. If we deploy this web part as a feature (like most do) in Visual Studios, that web part will be immediately available because it will deploy the feature to sharepoint, and turn on that feature. It also gives the ability to site administrators and farm administrators to turn off (or toggle) the feature. That way the web part will no longer be available.

[SharePoint Feature](http://msdn.microsoft.com/en-us/library/bb861828%28v=office.12%29.aspx) (2007 but still applicable)
Features make it easier to deploy managed solutions or site customizations to SharePoint. A feature can be just about anything, but more often than not it seems they are ASP.Net wrapped in a solution and deployed to SharePoint. (IMHO)

Check these out, see if they help:

MSDN overvew
<http://msdn.microsoft.com/en-us/library/office/ms460318(v=office.14).aspx>

Channel 9 video
<http://channel9.msdn.com/Shows/Devs4Devs/Building-Features-in-SharePoint-2010>

Step by step guide in blog
<http://weblogs.asp.net/sreejukg/archive/2011/10/27/developing-sharepoint-2010-features-using-visual-studio-2010.aspx>