added functionality to output the probabilities of the generated tokens #5164
see https://github.com/huggingface/transformers/issues/5164

running:

    from transformers import AutoTokenizer, GPT2LMHeadModel, AutoModelWithLMHead
    model_name = 'facebook/bart-base'
    print("load tokenizer")
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    print("load model")
    model = AutoModelWithLMHead.from_pretrained(model_name, pad_token_id=tokenizer.eos_token_id if tokenizer.pad_token_id is None else tokenizer.pad_token_id)
    tokenizer_kwargs = {"add_prefix_space": True}
    s1 = "Hello! Looks like you’re enjoying the discussion, but you haven’t signed up for an account yet. When you create"
    # s2 = "wow, that's cool"
    tokens = tokenizer(s1, return_tensors='pt', **tokenizer_kwargs, )
    # input_ids = tokenizer.encode(str, return_tensors='pt',  **tokenizer_kwargs)
    outputs, probs = model.generate(input_ids=tokens['input_ids'], return_probs=True, num_beams=3, num_return_sequences=3, max_length=100)
    outputs = model.generate(input_ids=tokens['input_ids'], num_beams=3,
                                    num_return_sequences=3, max_length=100)
Hi, is there any reason why this function not merged into the generate function? Hope to use this functionality to incorporate with reinforcement learning in my work.<|||||>Hey @ngoquanghuy99,

Sorry to answer only now. It is already possible to output probabilities as shown here: https://discuss.huggingface.co/t/generation-probabilities-how-to-compute-probabilities-of-output-scores-for-gpt2/3175/15?u=patrickvonplaten