Fix passive element segment GC roots' endianness#13230
Merged
fitzgen merged 3 commits intobytecodealliance:mainfrom Apr 29, 2026
Merged
Fix passive element segment GC roots' endianness#13230fitzgen merged 3 commits intobytecodealliance:mainfrom
fitzgen merged 3 commits intobytecodealliance:mainfrom
Conversation
Passive element segments were registered as GC roots by converting a `*mut ValRaw` to a `*mut VMGcRef`. Because we always store GC refs as little endian inside `ValRaw`, regardless of the target architecture's endianness, this cast is only valid on little endian systems. This bug was not exposed until the introduction of the copying collector in bytecodealliance#13107 The fix that this commit makes is to add another kind of GC root for `ValRaw`, where we can get and set the GC root using `ValRaw` internally, to ensure that the endianness (and incidentally also the GC ref offsets within the `ValRaw`) are matched up correctly between the GC root's definition and the collector's use of it. This brings us to three kinds of GC roots: Wasm stack roots, `VMGcRef` roots, and `ValRaw` roots. FWIW, I initially tried to make `VMGcRef` also always store its data as little endian, but this was a larger, more-invasive change and with feedback like bytecodealliance#13193 (comment) suggesting the use of `[u8; 4]` instead of `u32` to make the byte ordering explicit, we break `rustc`'s niche type optimizations (since `VMGcRef` is non-zero right now). I also investigated making `PassiveElementSegment` an `enum` or either funcrefs or externrefs, similar to what we do for `wasmtime::runtime::vm::Table`. This also led to an outsized amount of code churn and didn't feel like it was paying for itself. Ultimately, I abandoned these approaches, preferring the one taken in this commit instead.
fitzgen
force-pushed
the
passive-element-segment-gc-roots-endianness
branch
from
01300c0 to
705f1b8
Compare
alexcrichton
approved these changes
Apr 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Passive element segments were registered as GC roots by converting a
*mut ValRawto a*mut VMGcRef. Because we always store GC refs as little endian insideValRaw, regardless of the target architecture's endianness, this cast is only valid on little endian systems. This bug was not exposed until the introduction of the copying collector in#13107
The fix that this commit makes is to add another kind of GC root for
ValRaw, where we can get and set the GC root usingValRawinternally, to ensure that the endianness (and incidentally also the GC ref offsets within theValRaw) are matched up correctly between the GC root's definition and the collector's use of it. This brings us to three kinds of GC roots: Wasm stack roots,VMGcRefroots, andValRawroots.FWIW, I initially tried to make
VMGcRefalso always store its data as little endian, but this was a larger, more-invasive change and with feedback like #13193 (comment) suggesting the use of[u8; 4]instead ofu32to make the byte ordering explicit, we breakrustc's niche type optimizations (sinceVMGcRefis non-zero right now). I also investigated makingPassiveElementSegmentanenumor either funcrefs or GC refs, similar to what we do forwasmtime::runtime::vm::Table. This also led to an outsized amount of code churn and didn't feel like it was paying for itself. Ultimately, I abandoned these approaches, preferring the one taken in this commit instead.