Question: I am currently working on an application in WPF/C# for personal use. I am not a "classically trained" programmer. Simply a hobbyist that likes to code in his spare time. Is there any accepted approach to the progression of application development? I. E.; Make it work, add fault tolerance, create a gui, then performance optimization. Or maybe should I design the entire GUI first? Basically I am going to start a new project soon and would like to have some sort of "every program needs this" checklist.

Answer: 
I'd say first of all, as simple as it sounds, write (either on paper or in your head) what the app is actually going to do. (Most businesses can't even do this part!)

Knowing that, sketch on paper what you think the screens will look like.
For example, say you're writing a home budget program and you want a drop down for accounts, and a grid for transactions, etc. Knowing what the GUI looks like will help tremendously.

Then fill in the details.
The development lifecycle varies a lot. Variance is due to project size, team size, timeline, etc...

For small hobby projects, I usually follow this approach:

1. Specs: this could be a half page typed about what I think I'm doing, and maybe some diagrams to make it more clear.
2. Plan: Usually a document where I outline what I think major milestones will be, such as "complete proof of concept", "basic gui", "system logging", "successful CRUD operations"
3. Code: Try to meet the first milestone in 2., then possibly re-evaluate 2. Continue until project is done, or I get bored / distracted by something else (usually shinier than whatever I'm working).

This last step may also involve sub-steps such as data modelling (if it's a database app) or graphics design for icons (if the GUI needs fancy icons).