  Prompt-based techniques have demostrated great potential for improving the
few-shot generalization of pretrained language models. However, their
performance heavily relies on the manual design of prompts and thus requires a
lot of human efforts. In this paper, we introduce Genetic Prompt Search (GPS)
to improve few-shot learning with prompts, which utilizes a genetic algorithm
to automatically search for high-performing prompts. GPS is gradient-free and
requires no update of model parameters but only a small validation set.
Experiments on diverse datasets proved the effectiveness of GPS, which
outperforms manual prompts by a large margin of 2.6 points. Our method is also
better than other parameter-efficient tuning methods such as prompt tuning.

Computation and Language