Yes, I definitely agree – we have to walk before we run :’). I definitely want to stick to a plan we can accomplish in the allotted time (7 days) and happy with the proposal and excited to learn and work with others on it. I hope to collaborate with others in the future as a research project on this topic. I think this is a great first step.
Also nice to meet you all! :’)
Great to also hear that we have the option to create one big team or multiple teams!
I wonder if our big group (or each smaller team if we go that route) can train multiple architectures (e.g. both RoBERTa and T5) so that each of us can get the chance to learn/practice Jax. Again, if we have time, it might be great to fine-tune our pretrained model on benchmark code generation tasks e.g. CoNaLa and Shellcode_IA32 (self-plugging right here :’))
Hope this proposal makes it through!