Is baggage always offloaded when a passenger does not travel? For terrorism concerns I think many airlines offload checked in baggage if a passenger somehow does not end up travelling on a flight. 

How strictly are these rules followed? e.g. If a connecting passenger with a tight connection cannot make it to the gate in time will the airline actually send in a ramp guy into the baggage hold to remove specific pieces of baggage? How feasible is it to do thing when the bags are in pods or the Unit Load Devices? Do they have to remove the load device from the hold before they can extract a specific bag?

How does the gate delay of just waiting for the errant, late passenger compare to the time &amp; effort needed to find &amp; offload his bags? 

PS. Since bags do get misrouted I'm sure at least some bags fly without a passenger owning them on board.
Normally, baggage takes much longer to switch flights than self-loading cargo, so your scenario is highly unlikely.

In the event of a passenger missing and his/her baggage already on board, all bags need to be unloaded until the orphaned bag is found. When ULDs are used, those must be unloaded first and are then unpacked, mostly on the apron next to the aircraft.

Waiting for the missing passenger is an open-ended game. At some point, you must decide on a course of action with a fixed exit so you can move on. 

Normally, every airline is required to unload an orphaned bag. I am sure some are more strict about this than others.