Talking to myself: how I trained GPT2-1.5b for rubber ducking using my chat data
From anecdotal testing, using the 774M/1.5B GPT-2 models for anything less than <i>hundreds of megabytes of input data</i> will result in <i>worse</i> generation quality than using the smaller 124M/355M models.<p>The addiction to the larger GPT-2 models is IMO a trap.