Research · NVIDIA Technical Blog
24 GPU operators in Rust: agents benefited from a shared compiler language
NVIDIA described an agent-driven port of 24 TileGym operators from Python to Rust with nearly equal GPU-kernel execution speed. The distinctive feature was the ability to compare compiler intermediate code and verify each stage automatically. For teams maintaining custom compute operations, this changes what a local coding assistant needs around it.
What this means for a buyer
A coding chat and a verifiable GPU-kernel port need different environments. Here, reference implementations, a compiler, tests and measurements support the result. Buyers should assess the preparation cost of that combination for their own operators before increasing their assistant’s compute resources.
Near parity belongs to a specific measurement
NVIDIA’s team ported all 24 public TileGym operators to cuTile Rust: roughly 40 GPU kernels, including normalization, attention operations and MoE computation. The overall geometric-mean speed ratio against cuTile Python was 0.995. Measurements used NVIDIA DGX B200, one exclusive GPU per backend, 347 paired configurations and the best measurements across four CI runs. This is a strong result within a defined porting task. NVIDIA’s analysis.
The reported metric was CUPTI device time: execution time for the kernel itself on the GPU. User-visible elapsed time also includes launch and scheduling, particularly relevant for very short operations. The 0.995 ratio therefore describes compute kernels, rather than how quickly an agent writes software or how much faster an entire local server becomes. The authors measured wall-clock time but reported device time for the final operator comparison.
Why this translation is checkable
cuTile Python, Triton-TileIR and cuTile Rust converge on CUDA Tile IR, an intermediate representation of the program. The same tileiras compiler then optimizes operations and creates the GPU binary. A port reproduces the computation through another language interface. The team has a more precise reference than similar-looking source code: it can compare data-tile shapes, memory operations and reductions inside the compiler representation.
The main difficulty is specialization. Python implicitly adapts a kernel to call parameters; Rust requires those parameters in its signature. For example, a Python branch may disappear before compilation, while both Rust branches must type-check. One original kernel can therefore require multiple structural variants. The agent must reconstruct the program’s hidden contract rather than merely replace its syntax.
Errors are isolated before full integration
The process begins by analyzing a reference and producing analysis.json: variants, data types, tolerances, launch parameters and other conditions. The kernel writer works only on kernel.rs. An isolated Rust test checks the result without Python or the language bridge, followed by a Tile IR comparison. This sequence helps distinguish a computation error from a component-connection error before the complete extension exists.
The next stage adds the C ABI, the calling agreement that lets another language invoke Rust, and a Python wrapper. TileGym tests exercise actual shapes and data types. Only a successful machine-checkable verdict permits performance measurement. The authors emphasize that a plausible kernel can pass ordinary tests while losing an important attribute or using a different memory operation. IR comparison extends verification beyond the examples.
Rust does not remove hazardous interfaces
cuTile Rust extends Rust’s ownership model to GPU kernels and splits mutable outputs into disjoint pieces. However, some shipped ports use an unchecked interface because it reproduces the reference IR; migration to the safe interface continues. Beyond the C ABI boundary, an incorrect array stride can also silently corrupt data. The wrapper and interface consequently need their own checks.
In the described PyTorch connection, the launching layer neither copies nor allocates memory, takes no ownership of another component’s buffer and uses the caller’s CUDA stream. These details matter when integrating existing code. They also explain why mathematical correctness alone cannot establish a successful port: data shape, memory access and agreements between languages must remain intact.
The major investment is a prepared environment
The described package requires CUDA 13.1+, Rust 1.89+, the tileiras compiler and a Blackwell GPU for performance checking. The authors say a full conversion consumes tokens on the order of millions. The workflow caps retries and preserves a diagnosis when it cannot converge. Preparation therefore includes reference implementations, suitable tools and someone able to distinguish a translation defect from a compiler problem.
The result is most relevant to teams that maintain custom GPU operators. Shared IR made verifiable automation possible within this particular tool family; an arbitrary Python project does not automatically offer that foundation. The sensible choice is to budget a reproducible environment for one necessary operator first. That preparation will determine a local agent’s value before there is a reason to expand its server.
What to check before choosing
- Which custom GPU operators does the team regularly change and actually want to port?
- Who can investigate an IR divergence and a possible upstream compiler defect?
- How does the budget account for first-launch compilation and subsequent cache reuse?
- What token-spending limit applies to one completed conversion?
Buyer implications are our editorial interpretation. The publication does not establish the speed or compatibility of a particular catalogue system.