zero-shot results, a rigorous methodology was implemented. This involves the engagement of three independent prompt
engineers, each undergoing five rounds of refinement for zero-shot settings, followed by a series of post-processing
techniques to enhance result reliability. The best results are thus obtained from each of the three groups. We still
acknowledge the potential for refinement and improvement.
We share some internal training experiences, acknowledging the absence of robust validation. First, we observe that
a higher weight decay (e.g., 0.1 versus 0.01) generally improves performance in both pre-training and instruction-
DOCLLM