Question: I am currently working on an application in WPF/C# for personal use. I am not a "classically trained" programmer. Simply a hobbyist that likes to code in his spare time. Is there any accepted approach to the progression of application development? I. E.; Make it work, add fault tolerance, create a gui, then performance optimization. Or maybe should I design the entire GUI first? Basically I am going to start a new project soon and would like to have some sort of "every program needs this" checklist.

Answer: 
If this is something you're doing as a hobby, you can choose any development methodology you want.

Typically, for commercial development, there's some form of prototype made first (for WPF, SketchFlow with Blend is fantastic for this). This is typically required since you most often need to "sell" the concept, either to a client, management, etc.

However, if you're doing this on your own, you have the freedom to do things any way you wish. One comment, however. I would, personally, not thing of "fault tolerance" as separate from "making it work", however, since "fault tolerance" comes with testing, and unit testing as you go is much more effective...
What has worked for me in the past is the following (high level) sequence:

1. Understand and record the requirements (who, what and when) - use-cases, feature list, flowcharts, service level requirements - whatever works for you (I am partial to use-cases though).
2. Understand and record the design (how) - class diagrams, sequence diagrams, screen designs, API specifications - start high level and drill down (could start development before complete drill down)
3. Implement - start with API/stubs and unit tests, then fill in the code, update unit tests and execute unit tests as you go.
4. System test - test the components operating together and fix defects. Don't forget about performance testing (check that you have met your service level requirements from step 1).
5. Package, deploy and enjoy.