You can use affine_grid() followed by grid_sample() to perform warping. Here is the documentation : http://pytorch.org/docs/master/_modules/torch/nn/functional.html#grid_sample 1.0k .
In your case you already have the flow field, so appplying grid_sample(Video_frame, optical_flow) would suffice. However, the flow field should be in the range [-1,1] assuming normalized co-ordinates of the image/video.
Hope that helps