Safe SIMD in Rust, even on the inside

2026-06-19T00:00:00+00:00
unsafe</span> {</span></span>
    #</span>[</span>cfg</span>(</span>all</span>(</span>any</span>(</span>target_arch </span>=</span> "x86"</span>,</span> target_arch </span>=</span> "x86_64"</span>),</span> target_feature </span>=</span> "avx2"</span>))]</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
    #</span>[</span>cfg</span>(</span>all</span>(</span>any</span>(</span>target_arch </span>=</span> "x86"</span>,</span> target_arch </span>=</span> "x86_64"</span>),</span> target_feature </span>=</span> "sse"</span>,</span> not</span>(</span>target_feature </span>=</span> "avx2"</span>)))]</span></span>
    _mm_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
    #</span>[</span>cfg</span>(</span>all</span>(</span>target_arch </span>=</span> "aarch64"</span>,</span> target_feature </span>=</span> "neon"</span>))]</span></span>
    vaddq_f32</span>(</span>a</span>,</span> b</span>)</span></span>
}</span></span></code></pre>
Look at it. It’s hideous. And the whole thing is wrapped in unsafe</code>!</p>
And that’s a simplified example. It still doesn’t handle:</p>

Other common platforms: AVX-512, 32-bit ARM, WebAssembly</li>
Platforms without SIMD or obscure platforms like RISC-V</li>
Actually loading data like &[f32]</code> into a form that each intrinsic accepts</li>
Selecting the best implementation for the CPU it’s running on</li>
</ul>
Luckily, Rust provides many SIMD abstractions</a> that handle all of that for you and let you simply write a + b</code>.</p>
There is just one wrinkle. Inside, they’re still full of unsafe</code>. It wasn’t gone, just hidden. Vast quantities of it lurking just beneath the surface, getting screwed</a> up</a> occasionally</a>.</p>
Or rather, they were. Until now.</p>
Why do we even need ‘unsafe’?</i></a> 
</h2>
For the longest time you couldn’t get around wrapping the call to each intrinsic function such as _mm256_add_ps</code> into unsafe</code> because it is illegal to call one when it’s not available on the CPU you’re running on.</p>
So you had</em> to have some mechanism for tracking which instructions are needed for each intrinsic, and which instructions you have access to, and cross-referencing them to decide if it’s safe to call a given function.</p>
It was either tedious if done by hand or complex if done by a code generator, always error-prone, and required unsafe</code> around every intrinsic.</p>
This changed in Rust 1.87 when the compiler started tracking the required instruction sets itself, so you could write this:</p>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> add_avx2</span>(</span>a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span> // this is an avx2 intrinsic</span></span>
}</span></span></code></pre>
Look ma, no unsafe</code>!</p>
…yet.</p>
You still cannot write a + b</code> with this. The best you can do is this:</p>
unsafe</span> {</span> add_avx2</span>(</span>a</span>,</span> b</span>) }</span></span></code></pre>
This only shifts the unsafe</code> up a layer. You can call intrinsics inside functions annotated with the correct #[target_feature]</code> now, but there still has to be unsafe</code> somewhere in the chain.</p>
The other problem is more fundamental. You cannot put #[target_feature]</code> on the implementation of +</code> for your type, because +</code> must be available always. So no a + b</code> for us using this mechanism.</p>
Lemma: CPU feature tokens</i></a> 
</h2>
To understand how the final solution works, you first need to understand how CPU feature detection works.</p>
Normally, checking for a CPU feature like AVX2 is done at runtime using is_x86_feature_detected!("avx2")</code>. But we definitely don’t want to run this check every single time we add two numbers together — that would completely tank performance. We want to check it once</em>, and then prove to the compiler that it’s safe to use AVX2 instructions from that point on.</p>
Instead we can encode this proof into the type system using an unforgeable token</em>: a zero-sized type with a private inner field. The only</em> way to obtain this token is to call a function that performs the CPU feature check. If the check passes, the function hands you the token:</p>
pub struct</span> Avx2</span>(());</span></span>
</span>
fn</span> detect_avx2</span>()</span> -></span> Option</span><</span>Avx2</span>></span> {</span></span>
    if</span> is_x86_feature_detected!</span>(</span>"avx2"</span>) {</span></span>
        Some</span>(</span>Avx2</span>(()))</span></span>
    }</span> else</span> {</span></span>
        None</span></span>
    }</span></span>
}</span></span></code></pre>
And because it’s a zero-sized type, passing this token around has no runtime overhead. It exists purely as a compile-time proof.</p>
The upshot is that as long as you have an instance of the Avx2</code> struct, you can be sure that AVX2 instructions are available on the system.</p>
The key insight</i></a> 
</h2>
The compiler doesn’t know it, but this function is safe to call:</p>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> add_avx2</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
}</span></span></code></pre>
You can only</em> call this function if you have an Avx2</code> token, which you can only get if AVX2 instructions are available on the system.</p>
If we can explain to the compiler that this is valid (using unsafe</code>), we can write that unsafe</code> only once and reuse it everywhere.</p>
What we need is a macro which is safe to invoke:</p>
with_avx2!</span>(</span></span>
    fn</span> add_avx2</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
        _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
    }</span></span>
)</span></span></code></pre>
but expands into this behind the scenes:</p>
fn</span> add_avx2</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    // SAFETY: Avx2 is available according to the token,</span></span>
    // and we verified that the inner function is not an `unsafe fn`</span></span>
    unsafe</span> {</span> inner</span>(</span>token</span>,</span> a</span>,</span> b</span>) }</span></span>
</span>
    #</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
    fn</span> inner</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
        _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
    }</span></span>
}</span></span></code></pre>
Now if you use an intrinsic that isn’t in AVX2, the compiler will reject it!</strong></p>
We’ve just managed to provide a safe programming interface to SIMD intrinsics without any bespoke tracking of target features!</p>
Even though there is one unsafe</code> block still inside it, it’s encapsulated in a sound API, so you cannot misuse it to cause memory safety bugs. In that sense it is just like println!</code>, safely abstracting unsafe code.</p>
This way you only ever need to review and audit this one macro,</em> not hundreds upon hundreds of bespoke unsafe</code> blocks. And the only things we could possibly</em> screw up in the implementation are:</p>

Mapping the token to the wrong #[target_feature]</code></li>
Allowing an unsafe fn</code> to be called from a safe context</li>
</ol>
And both failure modes are quite easy to check for.</p>
So now we can call add_avx2(token, a, b)</code> without unsafe</code>, but that still doesn’t get us to a + b</code>. How do we solve that?</em></p>
Generics to the rescue</i></a> 
</h2>
We cannot annotate the implementation of a + b</code> with #[target_feature]</code> because it must be safe to call from anywhere. And we cannot pass a token into the function because it accepts a</code> and b</code> but not token</code>.</p>
But even if we could do that, it would make for a pretty ugly API. We want a + b</code> to always work and automatically use the best SIMD instructions without the user ever fussing with tokens.</p>
We can use generics to solve both problems at once: by defining an f32x8</code> type that’s generic over the available instruction sets, we can implement addition on it that both smuggles a token inside it and</em> creates a separate implementation for each SIMD instruction set!</p>
This is what it looks like:</p>
pub trait</span> Level</span> {}</span></span>
</span>
#</span>[</span>derive</span>(</span>Clone</span>,</span> Copy</span>)]</span></span>
pub struct</span> Avx2</span>(());</span></span>
impl</span> Level</span> for</span> Avx2</span> {}</span></span>
</span>
pub struct</span> f32x8</span><</span>L</span>:</span> Level</span>></span> {</span></span>
    // For simplicity we'll back this with an array in this example.</span></span>
    // In production code we use native SIMD types for the level.</span></span>
    data</span>:</span> [</span>f32</span>;</span> 8</span>],</span></span>
    // The smuggled token!</span></span>
    token</span>:</span> L</span>,</span></span>
}</span></span>
</span>
/// implementation of `a + b` for Avx2</span></span>
impl</span> std</span>::</span>ops</span>::</span>Add</span> for</span> f32x8</span><</span>Avx2</span>></span> {</span></span>
    type</span> Output</span> =</span> Self</span>;</span></span>
</span>
    fn</span> add</span>(</span>self</span>,</span> rhs</span>:</span> Self</span>)</span> -></span> Self</span>::</span>Output</span> {</span></span>
        // (type conversions abbreviated)</span></span>
        // Use the Avx2 token to call our safe wrapper</span></span>
        let</span> result</span> =</span> add_avx2</span>(</span>self</span>.</span>token</span>,</span> self</span>,</span> rhs</span>);</span></span>
        Self</span> {</span></span>
            data</span>:</span> store_m256</span>(</span>result</span>),</span></span>
            token</span>:</span> self</span>.</span>token</span>,</span></span>
        }</span></span>
    }</span></span>
}</span></span></code></pre>
And then we can just as easily make it work for any other instruction set, or when SIMD is not available at all:</p>
#</span>[</span>derive</span>(</span>Clone</span>,</span> Copy</span>)]</span></span>
pub struct</span> NoSimd</span>(());</span></span>
impl</span> Level</span> for</span> NoSimd</span> {}</span></span>
</span>
/// implementation of `a + b` when no SIMD is available</span></span>
impl</span> std</span>::</span>ops</span>::</span>Add</span> for</span> f32x8</span><</span>NoSimd</span>></span> {</span></span>
    type</span> Output</span> =</span> Self</span>;</span></span>
</span>
    fn</span> add</span>(</span>self</span>,</span> rhs</span>:</span> Self</span>)</span> -></span> Self</span>::</span>Output</span> {</span></span>
        let</span> result</span> =</span> std</span>::</span>array</span>::</span>from_fn</span>(</span>|</span>i</span>|</span> self</span>.</span>data</span>[</span>i</span>]</span> +</span> rhs</span>.</span>data</span>[</span>i</span>]);</span></span>
        Self</span> {</span></span>
            data</span>:</span> result</span>,</span></span>
            token</span>:</span> self</span>.</span>token</span>,</span></span>
        }</span></span>
    }</span></span>
}</span></span></code></pre>
We’ve just solved safety and</em> runtime instruction selection at once!</p>
Add a convenience function that gives you the best Level</code> available on the system, and you get pretty much the perfect API for SIMD!</p>
The ABI would like a word</i></a> 
</h2>
There is, unfortunately, a fundamental problem to writing a + b</code> and having it lower to SIMD instructions: function call overhead.</strong></p>
Calling a function is not free but pretty cheap — just a handful of CPU instructions. But a handful</em> of instructions is a lot more than one</em> instruction we’ve just used for implementing addition!</p>
So if there’s a function call in the way, addition performance will plummet. And performance is the entire point of using SIMD in the first place!</p>
The compiler is usually pretty good at erasing this overhead via inlining</a>. It basically copy-pastes the implementation of a function you called into the function calling it, so there’s no more function and no more overhead.</p>
But #[target_feature]</code> annotations throw a wrench in the works. The compiler cannot inline a function that has a #[target_feature]</code> annotation into one that doesn’t, because the required features are not available in it!</p>
And guess what cannot have a #[target_feature]</code> annotation? Yeah.</p>
So how do</em> we make a + b</code> work with SIMD?</p>
Inline it, inline it with fire!</i></a> 
</h2>
We cannot put #[target_feature]</code> on the function that implements a + b</code>, but we can</em> put it on the function that calls a + b</code>!</p>
Then we can use inlining</a> to get the implementation of a + b</code> copied into the function calling it, and it ends up in a #[target_feature]</code> context.</p>
So the call chain looks like this:</p>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> do_stuff</span>() {</span></span>
    // TODO: some computation</span></span>
    c</span> =</span> a</span> +</span> b</span>;</span></span>
    // TODO: some more computation</span></span>
}</span></span>
</span>
// which calls into...</span></span>
</span>
#</span>[</span>inline</span>(</span>always</span>)]</span> // Function body will be copied into the caller</span></span>
fn</span> add</span>(</span>self</span>,</span> rhs</span>:</span> Self</span>)</span> -></span> Self</span>::</span>Output</span> {</span></span>
    // Use the Avx2 token to call our safe wrapper</span></span>
    add_avx2</span>(</span>self</span>.</span>token</span>,</span> self</span>,</span> rhs</span>);</span></span>
    // return statement abbreviated</span></span>
}</span></span>
</span>
// which calls into...</span></span>
</span>
#</span>[</span>inline</span>]</span> // Function body will be copied into the caller if feasible</span></span>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> add_avx2</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
}</span></span></code></pre>
This works.</p>
You can add these annotations in the right places and abstract over the SIMD levels and you don’t even need macros</a>.</p>
The problem is that whenever you call a + b</code> on SIMD types, you have to do it from a function with either #[inline(always)]</code> or #[target_feature]</code> on it, otherwise the code still compiles but performance plummets.</p>
Wanna see for yourself? Open this example</a>, remove #[target_feature]</code> from it and watch the generated assembly turn into absolute horror show.</p>
I’m not sure what can be done about this. The limitation seems quite fundamental for any approach that implements a + b</code> with SIMD.</p>
The Struct Target Features RFC</a> solves this for add_avx2(token, a, b)</code> and add::<S: Simd>(token, a, b)</code> but I don’t see a path to a + b</code> just yet.</p>
Where does that leave us?</i></a> 
</h2>
Despite the inlining wrinkle inherent to all SIMD code, we’ve managed to provide a remarkably pleasant SIMD abstraction at a staggeringly low, never-before-seen level of unsafe code.</p>
You can find the production version of these ideas in fearless_simd</code></a> v0.5, available now</strong> in a package registry near you! And here’s a small example</a> to see how it all fits together in production.</p>
The macro used to implement it is also exposed publicly, so you can easily mix and match</a> high-level operations like a + b</code> and platform-specific intrinsics to take full advantage of the hardware.</p>
There is more than one unsafe</code> block in fearless_simd</code> because it also provides the functionality of safe_unaligned_simd</code></a> crate, but that too is done at a significantly lower</a> amount of unsafe code than the original.</p>
For me the barrier to using high-level SIMD abstractions was always the sheer amount of unsafe</code> they brought. It was scary and hard to justify.</p>
But now SIMD in Rust can be truly fearless.</p>
Acknowledgements</i></a> 
</h2>
I am shocked that I am the first to put this into production (as far as I can tell), because I’m certainly not the first person to think of this.</p>
CPU feature tokens are an old and common idea. The pulp</code></a> crate has been using them for years, but they relied on handwritten unsafe wrappers around intrinsics, and occasionally got them wrong</a>.</p>
Generating multiple implementations using generics is also an old idea. It is part of the original fearless_simd</code> concept</a> from 8 years ago. The simdeez</code></a> crate, which predates it, also seems to use something similar</a>.</p>
The key insight of combining tokens with a single safe wrapper that delegates to rustc</code> is not unique to me either. Just in the context of fearless_simd</code> crate, Raph Levien has experimented with it</a>, and Daniel McNab</a> created a more elaborate implementation</a> than mine months ago.</p>
Daniel’s approach allows fine-grained tracking of every individual CPU feature, as opposed to a handful of fixed CPU feature levels that fearless_simd</code> uses. It is more expressive, but came at the cost of complexity, and his approach never got merged because no other maintainer stepped up to review it. I still hope it will be published as a standalone crate someday.</p>
Thanks to Daniel and to Laurenz Stampfl</a> for reviewing all my PRs to fearless_simd</code>, they were big and the quick reviews are really appreciated!</p>
This article was originally published on my Medium</a>.</em></p>
Sergey "Shnatsel" Davidoff

The unreasonable effectiveness of LLMs for auditing Rust code

Methodology

Wait, isn’t Rust memory-safe?

Preventing false positives

Harness

Findings

Fixes

Reflections

Limitations

Safe SIMD in Rust, even on the inside

My old articles are on Medium
