Q:

How does OS detect a crash of a process

How does OS know that EIP is no longer a valid/legal instruction and that application has crashed? How does it know when to generate the crash dump data?

A:

On an x86-compatible processor, when EIP points to a page which does not have read permission, a page that is not mapped, an invalid instruction, or when a valid instruction tries to access a memory page without permission, or a page that is not mapped, or a divide instruction sees that the denominator is zero, or an INT instruction is executed, or a bunch of other things, it raises an exception. In the case of an exception occuring in protected mode when the current privilege level (CPL) is > 0, the following things occur:

Loads the values for SS and ESP from a memory section called the task state segment.
Pushes the values of SS, ESP, EFLAGS, CS and EIP onto the stack. The SS and ESP values are the previous ones, not the new ones from the TSS.
Some exceptions also push an error code onto the stack.
Gets the values for CS and EIP from the interrupt descriptor table and puts these values in CS and EIP.

Note that the kernel has set up these tables and segments in advance.
Then:

The kernel decides what to do with the exception. This depends on the specific kernel. Usually, it decides to kill your program. On Linux, you can override this default using signal handling and on Windows you can override it using Structured Exception Handling.

(This is not an exhaustive reference to x86 exception handling. This is a brief overview of the most common case.)

