title: CRUST-Bench: A Comprehensive Benchmark for C-to-safe-Rust Transpilation
summary: C-to-Rust transpilation is essential for modernizing legacy C code while
  enhancing safety and interoperability with modern Rust ecosystems. However, no
  dataset currently exists for evaluating whether a system can transpile C into
  safe Rust that passes a set of test cases. We introduce CRUST-Bench, a dataset
  of 100 C repositories, each paired with manually-written interfaces in safe
  Rust as well as test cases that can be used to validate correctness of the
  transpilation. By considering entire repositories rather than isolated
  functions, CRUST-Bench captures the challenges of translating complex projects
  with dependencies across multiple files. The provided Rust interfaces provide
  explicit specifications that ensure adherence to idiomatic, memory-safe Rust
  patterns, while the accompanying test cases enforce functional correctness. We
  evaluate state-of-the-art large language models (LLMs) on this task and find
  that safe and idiomatic Rust generation is still a challenging problem for
  various state-of-the-art methods and techniques. We also provide insights into
  the errors LLMs usually make in transpiling code from C to safe Rust. The best
  performing model, OpenAI o1, is able to solve only 15 tasks in a single-shot
  setting. Improvements on CRUST-Bench would lead to improved transpilation
  systems that can reason about complex scenarios and help in migrating legacy
  codebases from C into languages like Rust that ensure memory safety. You can
  find the dataset and code at https://github.com/anirudhkhatry/CRUST-bench.
paper_authors._id: 6807bf3e70a0cec724b8a011
paper_authors.name: Anirudh Khatry
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-23T08:30:04.897Z
paper_authors._id: 6697abd4be7ce6de07140e72
paper_authors.avatarUrl: /avatars/1598decabd9975d755082c8fb4e5962d.svg
paper_authors.fullname: Anirudh Khatry
paper_authors.type: user
paper_authors.user: anirudhkhatry
paper_authors._id: 6807bf3e70a0cec724b8a012
paper_authors.name: Robert Zhang
paper_authors._id: 6807bf3e70a0cec724b8a013
paper_authors.name: Jia Pan
paper_authors._id: 6807bf3e70a0cec724b8a014
paper_authors.name: Ziteng Wang
paper_authors._id: 6807bf3e70a0cec724b8a015
paper_authors.name: Qiaochu Chen
paper_authors._id: 6807bf3e70a0cec724b8a016
paper_authors.name: Greg Durrett
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-24T11:56:46.592Z
paper_authors._id: 65be9918b54ab5b37d1b67a7
paper_authors.avatarUrl: /avatars/9953707affb6881724c8efb2abf0c668.svg
paper_authors.fullname: Greg Durrett
paper_authors.type: user
paper_authors.user: gregdurrett
paper_authors._id: 6807bf3e70a0cec724b8a017
paper_authors.name: Isil Dillig
paper_title: CRUST-Bench: A Comprehensive Benchmark for C-to-safe-Rust Transpilation
paper_summary: C-to-Rust transpilation is essential for modernizing legacy C code while
  enhancing safety and interoperability with modern Rust ecosystems. However, no
  dataset currently exists for evaluating whether a system can transpile C into
  safe Rust that passes a set of test cases. We introduce CRUST-Bench, a dataset
  of 100 C repositories, each paired with manually-written interfaces in safe
  Rust as well as test cases that can be used to validate correctness of the
  transpilation. By considering entire repositories rather than isolated
  functions, CRUST-Bench captures the challenges of translating complex projects
  with dependencies across multiple files. The provided Rust interfaces provide
  explicit specifications that ensure adherence to idiomatic, memory-safe Rust
  patterns, while the accompanying test cases enforce functional correctness. We
  evaluate state-of-the-art large language models (LLMs) on this task and find
  that safe and idiomatic Rust generation is still a challenging problem for
  various state-of-the-art methods and techniques. We also provide insights into
  the errors LLMs usually make in transpiling code from C to safe Rust. The best
  performing model, OpenAI o1, is able to solve only 15 tasks in a single-shot
  setting. Improvements on CRUST-Bench would lead to improved transpilation
  systems that can reason about complex scenarios and help in migrating legacy
  codebases from C into languages like Rust that ensure memory safety. You can
  find the dataset and code at https://github.com/anirudhkhatry/CRUST-bench.
paper_ai_summary: A dataset, CRUST-Bench, evaluates the ability of large language models to transpile C to safe Rust, highlighting the challenges in maintaining idiomatic patterns and memory safety.
paper_ai_keywords: CRUST-Bench large language models LLMs C-to-Rust transpilation safe Rust idiomatic Rust memory safety dependencies functional correctness single-shot setting