Question: How a CAN transceiver detects the 5 successive high or low bits and inserting the opposite bit as stuff bit ?
what is the logic behind the bit stuffing process?. 

I am sure that, it's not handled in CAN driver software side.

Answer: 
Bit stuffing is on the interface, not the transceiver, like how there is also hardware in the interface to calculate the CRC and various other errors without ever having to touch the main processor, if it sees the successive bits, it inserts the inverted non data bit,

The transceiver remains just a level converter,
The hardware looks at the transmitted or received bits and can count them. Transmitting is easy, insert extra stuff bit when doing the line encoding. Receiving more than 5 same bits before line decoding to remove stuffed bits results into receive error.

The point is to have a line code with transition at least after 5 same bits to keep the receive bit clock in sync with the data stream. This allows for the devices to have cheap low precision systen clocks that have more tolerance. If this was not implemented, it would be difficult to keep track of how many same bits have been received, unless the system clocks are very precise, which is more expensive. Same thing why UART transmission has a start bit, to synchronize the reception of the next data, parity and stop bits.