It’s simple, a prompt is the input you give to the transformer, hinting at what the machine should do but most of the time not actually stating it.
Engineering of a prompt is trying different wording to make sure the machine understands what to do.
For example: “The English translation of [PHRASE] is [RESULT]”, Here, replace phrase by anything, and let GPT2 complete the result. This is an example of a prompt.
Some people call this “Prompt learning” as well.
As for what model to use, maybe try GPT-J by EleutherAI, sorry I’m not sure of the memory requirements.
Also, just so you know, if your project is open source you can apply for the TRC of Google Cloud, which will give you free TPUs, so you can use bigger models.