Mp4 Video Container and the Codecs use with it I have started to learn about mp4 file format, and I just learned that mp4 is nothing but just a container that defines the structure of the video file, and that it has different codecs inside it for audio and video, as well as other data like metadata and subtitles.

I have been searching around for a while now but couldn't find so much on the basics of mp4 containers and the working and complete structure explanation of it.

So far I have read this and this, but there is not enough information specifically about mp4 container.

My question is,

There are so many Audio/Video Codecs available which are used, but which Codecs are used with the mp4 container ?

Can we use any Codec with the mp4 container ? because there are like dozen different Codecs and how one would know which can be used with the mp4 container ?, 

and is there a book or a paper on Mp4 container structure and explanation ?
The Wikipedia article on MPEG-4 is a great start as the MP4 file specification is part of the MPEG-4 spec.  Specifically version 2 of MP4 is MPEG-4 Part 14. While not free, you can purchase copies of the ISO spec under ISO# 14496-14:2003.  A preview with some detail is available from the ISO here.  

It is designed to contain any of the various MPEG video formats, including MPEG, MPEG-2, MPEG-4 Part 2 (generic MPEG-4) and, probably most commonly, MPEG-4 part 10 (H.264).  It can theoretically support almost any format of video stream, however those 4 are the most widely supported for the file type.

The full list of officially registered codec types is available from the MP4 Registration Authority here.