Why is Target Display Mode via Thunderbolt so unreliable for me? I frequently want to display my MacBook Pro 15" Retina with my iMac Mid-2011, in Target Display Mode via a Thunderbolt cable.

TDM often fails on first attempt and just shows a black screen on the iMac. Getting TDM to work properly often requires multiple attempts to disconnect and reconnect the cable and going out and in of TDM again with fn + F2.

Also, the faintest touch on the cable can inadvertedly make it lose proper connection and have the iMac only display a black screen.

It's also erratic sometimes in it's way of handling audio, where sometimes it will be possible to change audio volume with the function keys, and sometimes not.

Is this behaviour "normal" (and to be expected) or is there something wrong with my cable?

Or is it that my iMac Mid-2011 that uses Thunderbolt v1 is really too slow to handle the video input? When it's actually working, it works just fine without stutters of any kind.
If you can touch the cable and have it drop in and out, I would say you either have a bad cable or bad connector (or both?) on one of the computers.

Try your setup with a different cable and see if you get the same result.

Good luck!