Recording and playing video with variable frame rate We are using a streaming camera to record and to play back surveillance video. 

This camera sends a RTSP stream which will be recorded by our application as a H264 video.

In our application, the H264 videos will be stored in a MP4 container and will be published on a HTTP-server (Apache2). The stored videos can now be watched with any HTML5 web browser. So far, so good! 

Our problem is, that the camera sends his frames with a variable frame rate. This is not a problem as long we watch the stream live. But when we watch the recorded H264/MP4 videos the time begins to drift. This means because of the variable frame rate, sometimes the video will be played too fast and the time is not correct!

So, does someone have experience with H264 videos with variable frame rate (VFR)? How can we play these VFR videos without time drift? Is it possible to convert VFR video to video with constant frame rate?

(All tested players like VLC seems to have the same problem...)

Here the output of mediainfo:

Complete name                            : VIDEO_REPLICA_12.h264
Format                                   : AVC
Format/Info                              : Advanced Video Codec
File size                                : 249 MiB
Overall bit rate mode                    : Variable

Video
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3.2
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 1 frame
Format settings, GOP                     : M=1, N=15
Bit rate mode                            : Variable
Maximum bit rate                         : 20.0 Mbps
Width                                    : 1 280 pixels
Height                                   : 800 pixels
Display aspect ratio                     : 1.600
Frame rate mode                          : Variable
Standard                                 : NTSC
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Are you sure the files your recorded have indeed a variable framerate? It sounds like you recorded with a constant framerate (even though you get a variable frame rate video feed) which introduced this "artifact" and the players then of course play it back with a constant frame rate even though they shouldn't.

To properly convert variable frame rate footage into constant frame rate footage you can reside to Handbrake. It's an easy to use tool for h264 encoding that lets you choose to convert your video to a constant frame rate.