Code Quality Analysis of Translations from C to Rust

C/C++ is a prevalent programming language. Yet, it suffers from significant memory and thread-safety issues. Recent studies have explored automated translation of C/C++ to safer languages, such as Rust. However, these studies focused mostly on the correctness and safety of the translated code, which are indeed critical, but they left other important quality concerns (e.g., performance, robustness, and maintainability) largely unexplored. This work investigates strengths and weaknesses of three C-to-Rust translators, namely C2Rust (a transpiler), C2SaferRust (an LLM-guided transpiler), and TranslationGym (an LLM-based direct translation). We perform an in-depth quantitative and qualitative analysis of several important quality attributes for the translated Rust code of the popular GNU coreutils, using human-based translation as a baseline. To assess the internal and external quality of the Rust code, we: (i) apply Clippy, a rule-based state-of-the-practice Rust static analysis tool; (ii) investigate the capability of an LLM (GPT-4o) to identify issues potentially overlooked by Clippy; and (iii) perform a manual analysis of the issues reported by Clippy and GPT-4o. Our results show that while newer techniques reduce some unsafe and non-idiomatic patterns, they frequently introduce new issues, revealing systematic trade-offs that are not visible under existing evaluation practices. Notably, none of the automated techniques consistently match or exceed human-written translations across all quality dimensions, yet even human-written Rust code exhibits persistent internal quality issues such as readability and non-idiomatic patterns. Together, these findings show that translation quality remains a multi-dimensional challenge, requiring systematic evaluation and targeted tool support beyond both naive automation and manual rewriting.

💡 Research Summary

The paper investigates the multi‑dimensional quality of Rust code produced by three automated C‑to‑Rust translation techniques—C2Rust (a pure transpiler), C2SaferRust (a transpiler followed by LLM‑guided refactoring), and TranslationGym (an LLM‑direct translation framework)—and compares them against a human‑written Rust baseline. The authors select seven utilities from GNU coreutils as a realistic, diverse testbed and translate each program with all four approaches.

To evaluate quality beyond functional correctness, the authors design a taxonomy of 18 issue categories that capture both internal quality (naming conventions, documentation, readability, idiomatic usage, redundancy, etc.) and external quality (memory safety, arithmetic bugs, performance, runtime panic risk, thread safety, error handling, etc.). Each category is aligned with ISO/IEC 25010 quality characteristics, providing a bridge between Rust‑specific linting and general software quality models.

The empirical methodology consists of three complementary analyses: (1) running Clippy, the de‑facto Rust linter, on every translated version; (2) prompting GPT‑4o to act as a “semantic linter” that can spot problems Clippy may miss, especially in non‑idiomatic, C‑style Rust; and (3) a manual, function‑level inspection of the warnings to understand root causes, interactions, and trade‑offs. Clippy’s 803 lint rules are first mapped to the taxonomy, allowing aggregated comparison across techniques. GPT‑4o is given a standardized prompt that asks it to classify each identified issue into one of the 18 categories, ensuring a consistent comparison with Clippy’s output.

Results reveal systematic trade‑offs. C2Rust faithfully reproduces the original C structure, leading to a high count of unsafe blocks, naming violations, and readability problems, but it preserves functional behavior. C2SaferRust reduces the number of unsafe blocks by having an LLM rewrite them into safer abstractions; however, this process introduces code duplication, increased cyclomatic complexity, and occasional performance regressions. TranslationGym, which translates functions directly with LLM assistance and iteratively validates with compiler feedback, produces the most idiomatic Rust overall, yet it still generates runtime‑panic risks (e.g., unchecked unwraps), subtle thread‑safety hazards, and occasional performance slow‑downs due to overly defensive patterns.

The human‑written Rust baseline scores highest on safety‑related external dimensions (memory safety, error handling) and exhibits fewer runtime panic risks, but it is not flawless: documentation gaps, non‑idiomatic naming, and some redundant code persist, indicating that even expert developers leave internal quality issues.

A key finding concerns the limitations of Clippy: because its lints target idiomatic Rust, many unsafe‑pointer patterns introduced by the translators escape detection, whereas GPT‑4o’s contextual reasoning surfaces these hidden risks. The combined toolchain therefore provides a more complete picture of translation quality.

From these observations the authors draw several implications. First, automated translation can improve specific quality aspects (e.g., reducing unsafe memory usage) but often does so at the expense of others (e.g., performance, readability). Second, a hybrid evaluation pipeline that includes rule‑based static analysis, LLM‑based semantic checks, and human review is essential for reliable assessment. Third, the proposed taxonomy and the 18‑category framework constitute a reusable benchmark for future translation tools and can guide the design of more balanced translators. Finally, the study suggests that automated translators are best viewed as assistive tools that accelerate development while still requiring expert oversight, rather than as replacements for human expertise.