title: CodeV: Empowering LLMs for Verilog Generation through Multi-Level
    Summarization
summary: The increasing complexity and high costs associated with modern processor
  design have led to a surge in demand for processor design automation.
  Instruction-tuned large language models (LLMs) have demonstrated remarkable
  performance in automatically generating code for general-purpose programming
  languages like Python. However, these methods fail on hardware description
  languages (HDLs) like Verilog due to the scarcity of high-quality instruction
  tuning data, as even advanced LLMs like GPT-3.5 exhibit limited performance on
  Verilog generation. Regarding this issue, we observe that (1) Verilog code
  collected from the real world has higher quality than those generated by LLMs.
  (2) LLMs like GPT-3.5 excel in summarizing Verilog code rather than generating
  it. Based on these observations, this paper introduces CodeV, a series of
  open-source instruction-tuned Verilog generation LLMs. Instead of generating
  descriptions first and then getting the corresponding code from advanced LLMs,
  we prompt the LLM with Verilog code and let the LLM generate the corresponding
  natural language description by multi-level summarization. Experimental results
  show that CodeV relatively surpasses the previous open-source SOTA by 14.4%
  (BetterV in VerilogEval) and 11.3% (RTLCoder in RTLLM) respectively, and also
  relatively outperforms previous commercial SOTA GPT-4 by 22.1% in VerilogEval.
paper_authors._id: 6698a542d723c35c0ec09743
paper_authors.name: Yang Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-18T09:05:13.327Z
paper_authors._id: 646f6231a6a58aa29507091a
paper_authors.avatarUrl: /avatars/14454308b13968cca11d8629350ca14d.svg
paper_authors.fullname: zhaoyang
paper_authors.type: user
paper_authors.user: yang-z
paper_authors._id: 6698a542d723c35c0ec09744
paper_authors.name: Di Huang
paper_authors._id: 6698a542d723c35c0ec09745
paper_authors.name: Chongxiao Li
paper_authors._id: 6698a542d723c35c0ec09746
paper_authors.name: Pengwei Jin
paper_authors._id: 6698a542d723c35c0ec09747
paper_authors.name: Ziyuan Nan
paper_authors._id: 6698a542d723c35c0ec09748
paper_authors.name: Tianyun Ma
paper_authors._id: 6698a542d723c35c0ec09749
paper_authors.name: Lei Qi
paper_authors._id: 6698a542d723c35c0ec0974a
paper_authors.name: Yansong Pan
paper_authors._id: 6698a542d723c35c0ec0974b
paper_authors.name: Zhenxing Zhang
paper_authors._id: 6698a542d723c35c0ec0974c
paper_authors.name: Rui Zhang
paper_authors._id: 6698a542d723c35c0ec0974d
paper_authors.name: Xishan Zhang
paper_authors._id: 6698a542d723c35c0ec0974e
paper_authors.name: Zidong Du
paper_authors._id: 6698a542d723c35c0ec0974f
paper_authors.name: Qi Guo
paper_authors._id: 6698a542d723c35c0ec09750
paper_authors.name: Xing Hu
paper_authors._id: 6698a542d723c35c0ec09751
paper_authors.name: Yunji Chen
paper_title: CodeV: Empowering LLMs for Verilog Generation through Multi-Level
    Summarization
paper_summary: The increasing complexity and high costs associated with modern processor
  design have led to a surge in demand for processor design automation.
  Instruction-tuned large language models (LLMs) have demonstrated remarkable
  performance in automatically generating code for general-purpose programming
  languages like Python. However, these methods fail on hardware description
  languages (HDLs) like Verilog due to the scarcity of high-quality instruction
  tuning data, as even advanced LLMs like GPT-3.5 exhibit limited performance on
  Verilog generation. Regarding this issue, we observe that (1) Verilog code
  collected from the real world has higher quality than those generated by LLMs.
  (2) LLMs like GPT-3.5 excel in summarizing Verilog code rather than generating
  it. Based on these observations, this paper introduces CodeV, a series of
  open-source instruction-tuned Verilog generation LLMs. Instead of generating
  descriptions first and then getting the corresponding code from advanced LLMs,
  we prompt the LLM with Verilog code and let the LLM generate the corresponding
  natural language description by multi-level summarization. Experimental results
  show that CodeV relatively surpasses the previous open-source SOTA by 14.4%
  (BetterV in VerilogEval) and 11.3% (RTLCoder in RTLLM) respectively, and also
  relatively outperforms previous commercial SOTA GPT-4 by 22.1% in VerilogEval.