How to avoid cramped/compressed breathing? Sometimes after a heavy effort I get into this compressed breathing which is very hard to break out of.

Heres what I mean with cramped breathing:
Imagine breathing with the top abdominal (1st row of rectus abdominis) muscles cramped up and pressing the air our instead of "belly breathing".

If I concentrate on breathing correctly I can make it go away temporarily but when the hunger for fresh air is high I quickly get back into this bad habit.

Its not painful or anything (this may sound worse than it is, but it is definetly not a medical issue) but I feel I cannot get enough air into my lungs this way and it makes me gasp for air and my breathing gets shorter and shorter until I have to slow down.

I thought that maybe my bike position may be at fault for this but on paper everything fits. 

How would you explain  this and help me improve my breathing ?
I have tried belly breathing but as I said, I kind of forget about it.
I see you are aware of horizontal breathing vs vertical breathing - the former is "belly breathing" as you name it, the latter is the shallow type with shoulders vertical movement involved (hence less effective).

Perhaps this helps: on my firefighter training we've learned how to cope with hyperventilation - make sure you exhale deeply. The body will take care for inhale itself provided the lungs are properly emptied.

Good luck!