Question about LLM
#4
by
Thisusernamealreadyexists00
- opened
Hello 😊, first I would like to ask you, if it's possible to take existing open LLM like some small llama, and implement HRM to it and pretrain or train it, and if yes, could we please implement it in colab or something please? 😊
and the second thing less important, if you want colaborative training in the google colab, training loop section the line should probably be
"for epoch in range(start_epoch, start_epoch + NUM_EPOCHS):"
otherwise it just skips the loop because start_epoch is something like 35 and num epochs is 5.
or maybe
"for epoch in range(0, NUM_EPOCHS):".
Yes you can take an existing small llama (or similar open LLM), add HRM blocks, and pretrain or fine-tune it. And thanks for the reccomendation.