Hi @bala1802, since the TextGenerationPipeline accepts a pretrained model as an argument perhaps you can adapt the following for your use case:
from transformers import AutoModelWithLMHead, AutoTokenizer, AutoConfig, TextGenerationPipeline

model_ckpt= "gpt2"
# override default parameters here, eg output the hidden states
config = AutoConfig.from_pretrained(model_ckpt, output_hidden_states=True)
model = AutoModelWithLMHead.from_pretrained(model_ckpt, config=config)
tokenizer = AutoTokenizer.from_pretrained(model_ckpt)

pipe = TextGenerationPipeline(model=model, tokenizer=tokenizer)

You might not even need the TextGenerationPipeline class since I think that this is equivalent to instantiating the pipeline as follows:
pipe = pipeline('text-generation', model=model, tokenizer=tokenizer)