Hey @pritam
You can pass output_attentions=True to forward  method, and it’ll return attentions. Attnetions are last element of the returned Tuple or if you are on master branch, then you can also pass return_dict=True   and call .attentions on the returned object.
output = model(input_ids, output_attentions=True, return_dict=True)
output.attentions # this will give the attentions