Llm, AI, Data Science, Machine Learning, Programming.
and with a low LoRA rank value, so there’s a lot of room for improvements. Here are some tips: Increasing the size of the fine-tuning dataset Training for more steps (epochs) Setting a higher LoRA rank Modifying the hyperparameter values such as learning_rate and weight_decay. Try using the larger