Question: How do I data mine a pile of text to get keywords by usage? ("Jacob Smith" or "fence")

And is there a software to do this already? even semi-automatically, and if it can filter out simple words like "the", "and", "or", then I could get to the topics quicker.

Answer: 
The general algorithm is going to go like this:

```

- Obtain Text
- Strip punctuation, special characters, etc.
- Strip "simple" words
- Split on Spaces
- Loop Over Split Text
    - Add word to Array/HashTable/Etc if it doesn't exist;
       if it does, increment counter for that word

```

The end result is a frequency count of all words in the text. You can then take these values and divide by the total number of words to get a percentage of frequency. Any further processing is up to you.

You're also going to want to look into [Stemming](http://en.wikipedia.org/wiki/Stemming). Stemming is used to reduce words to their root. For example `going => go`, `cars => car`, etc.

An algorithm like this is going to be common in spam filters, keyword indexing and the like.
You did not specify a technology you're working with, so I guess a shell script is also a possibility.

I've always been impressed by the word frequency analysis example in the [Advanced Bash-Scripting Guide](http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/textproc.html) (12-11)

The following for example fetches a book from project Gutenburg and writes out a word frequency analysis 'report':

```
wget http://www.gutenberg.org/files/20417/20417-8.txt -q -O- | 
sed -e 's/\.//g'  -e 's/\,//g' -e 's/ /\
/g' | tr 'A-Z' 'a-z' | sort | uniq -c | sort -nr > output.txt

```

Should be extendable to exclude words from a 'common' list (the, and, a...) etc.