Question: We currently have a 'global catch all exceptions' section in our application. When an uncaught exception is thrown, the stack trace is displayed and the application continues running.

More often than not, this leaves the state of the application invalid going forward. Especially with NullReferenceExceptions, and threading exceptions being the cause.

I decided to have the application log the exception in the 'global' section and shutdown and restart. This was met with criticism from management - who asserted that the user should choose to select whether or not to restart under these conditions, since it never shut down before. (*Though I tried my best to explain that the issue was allowing the app to continue running in the first place)*.

I am looking for things to watch out for now, and specific approaches I can take to handle code that is now allowed to continue to run while in an invalid state.

Answer: 
Gee this sounds familiar. 

I was once the manager for a group that had a large application which had a few unhandled exceptions that were eventually caught be a global catch-all-and-display-the-world-has-ended handler. By default this used to allow the application to keep running.

Your point about application state in this case is the same one I made: the application should not continue running because its internal state is unknown, and most likely suspect (after all that's how an exception came to be raised in the first place).

I wanted the application modified so that the user could save data and then the application would exit - no opportunity to continue. This was met by fierce resistance, mainly from developers who argued that it had continued for a long time and there had been minimal damage - just lots of complaints. In the end I had to back down - the user was allowed to continue but dialogs were changed to strongly suggest to users that they should exit and restart. I still think this was wrong.

In then end, every exception that is seen needs to be analysed to find the underlying cause, and appropriate fixes, code changes, or whatever need to be applied so that exceptions don't happen - or if they do they are caught as part of normal program flow and locally handled. 

(Sometimes, sadly, exceptions are propagated out of things like libraries. One of the rules I was taught a long, long time ago was: "do not ever rely on exceptions for normal program flow". It seems this is not always regularly practiced any more.)

I feel your pain - but in the end, the principle is a simple one and you should be sticking to it: When the last chance exception handler is fired, things are really sick. The program should exit. Doing otherwise gives users the misleading impression that they can continue working, but internally the program has an unknown or broken state. Continuing to work only leads to things getting progressively worse. In the end this translates into loss of customer satisfaction. Customers are also pissed off by a program that raises an exception and exits - if there is unsaved program state you should try and save it, perhaps in a manner that makes it clear that what is saved may not be reliable. At least you reduce the chance of data loss. But continuing to operate... bad bad move. You have to choose the lesser of 2 evils.
I can understand the point that your application managers are trying to make, that simply shutting down or restarting immediately upon encountering an unexpected error can confuse a user. You always want to give the user information about what happened and if possible give them a choice on how they would like to handle it.

You may also want to try and localize the problem to a particular component that may perhaps be reloaded or re-synced so as to prevent other working areas of the application from needing to be brought down?

On the other hand, if by them continuing to work and possibly corrupting persisted data which could cause future problems for themselves or others, OR if this bad or corrupted data needs manual intervention to clean up in some way then you have a case for shutting the application down as soon as possible. Just make sure that it is informative and graceful (... as graceful as an application crash could possibly be :)

Perhaps your efforts are better focused on stabilizing the application to where unexpected exceptions occur far less, and expected exceptions occur far greater.