I have a dataset with about 15,000 documents, containing about 7GB of uncompressed text, which I’m hoping to use for fine-tuning the Megatron-11b model 6, but before I get started, I’d like to get a rough estimate of the cost.
If I train in the cloud, I’ll probably use AWS P3 2 or G4 1 instances. Roughly how many instance-hours would I expect to need, to complete the fine-tuning?
At a certain price point, I might just decide to buy my own hardware, and do the training on-prem… What the minimum specs of hardware I’d need to be able to perform this task, without running out of memory? The pre-trained Megatron_11b model is 19GB of data… So does that mean I could perform model fine-tuning with only a single 24GB graphics card? Or would I need more than that?