These cards do not possess the memory capacity to fine-tune GPT-2. For other models that require less, the 6GB version of the RTX 1060 would be an affordable card to practice with. Tim Dettmers at least has a bench mark for it:
                    
            
           11


If this is out of your budget, you likely will be better off settling for free cloud computing services like Google Colab.