Gamma and beta are trainable parameters. Have a look at the original paper 6. It’s used to allow the layer to “learn” another normalization or even recover the original activations.
You can disable these parameters by setting affine=False in the BatchNorm layer.
During evaluation, the running_mean and running_var together with gamma and beta are used to normalize the data. The running stats were calculated during the training from the batch statistics. The momentum defines, how much of the current statistics (mean and var) are added to the history of these values.
The doc says:

x̂ new=(1−momentum)×x̂ +momemtum×xt, where x̂  is the estimated statistic and xt is the new observed value.

Let me know, if something is still unclear.