How does one do mindfulness of breathing with relaxation? In this lecture, by B. Alan Wallace, of mindfulness of breathing with relaxation, it has been instructed that I have to release all thoughts, relax every part of the body and let go of all involuntary thoughts during exhalation. 

Now what exactly is meant by releasing all thoughts with exhalation? What should I do during exhalation? Does it mean that mind will go blank during exhalation? If yes, then how is it possible?
Briefly, "let go" here means you turn your attention to something else (like the room in front of you) - and then whatever thoughts you had the moment before dissolve by themselves.

Normally our attention is dragged around by thoughts and experiences. Being able to disengage from them, and either hold on firmly to one thing, or to switch around at will, or to kinda suspend mid-air as thoughts and experiences flow by - is a useful skill to have.

Specifically in this case you're learning to disengage from whatever particular thoughts you have at the moment - to disidentify and to stop feeding.

It doesn't mean your mind goes blank, since you still have experiences of your body and surroundings, exhaling etc., - and you have an overall idea of what it is that you are doing (meditating, disengaging). So all mental functions keep on going, it's just that you are learning to gently let go of mental chatter, daydreaming, worrisome replays etc.

(no reference for this: I'm summarizing broadly based on what I was taught by the live Zen Master, what I read in books, and on my own meditation experience)