Model,Agg S2R,Agg MC,Agg VerilogEval S2R,Agg VerilogEval MC,Agg RTLLM,Agg VeriGen
DeepSeek R1,74.84,75.51,77.01,77.81,68.06,54.4
Llama 3.1 405B,49.72,42.8,53.98,42.92,36.43,41.67
Llama 3.(1-3) 70B,39.0,38.49,38.64,37.45,40.12,48.05
Qwen2.5 72B,49.23,48.82,49.17,51.22,49.45,26.75
Qwen2.5 32B,50.58,40.73,50.53,41.85,50.71,30.46
StarChat2 15B v0.1,39.04,38.9,37.45,37.69,44.0,49.99
DeepSeek R1 Distill Qwen 14B,22.98,23.61,23.21,23.47,22.27,24.91
CodeLlama 70B,31.46,31.29,34.17,29.8,22.99,44.96
QwenCoder 2.5 32B,42.53,43.71,42.27,43.96,43.33,41.4
DeepSeek Coder 33B,25.71,36.47,19.49,37.25,45.11,29.29
QwenCoder 2.5 14B,36.75,38.49,35.61,39.03,40.33,33.55
OpenCoder 8B,31.13,34.76,27.12,34.55,43.63,36.67
QwenCoder 2.5 7B,13.86,32.31,6.31,31.75,37.41,37.47
"DeepSeek Coder 6,7B",31.6,30.03,28.69,30.41,40.67,26.61
RTLCoder Mistral,21.86,27.2,22.73,26.21,19.15,36.3
RTLCoder DeepSeek,32.21,37.6,31.75,37.47,33.64,38.81
OriGen,37.22,41.29,46.0,41.97,9.82,35.07
HaVen-CodeQwen,41.66,46.09,42.97,46.57,37.55,41.74
CodeV-CL-7B,28.19,35.7,25.75,35.39,35.79,38.53
CodeV-QW-7B,20.79,47.26,18.73,50.28,27.23,19.55
CodeV-DS-6.7B,18.19,44.1,14.28,47.05,30.39,17.03