Skip to content

Expand Rust AEL fast path with native control lowering#2

Open
HeshamHM28 wants to merge 3 commits into
codeflash-ai:mainfrom
HeshamHM28:perf/rust-ael-native-control-lowering
Open

Expand Rust AEL fast path with native control lowering#2
HeshamHM28 wants to merge 3 commits into
codeflash-ai:mainfrom
HeshamHM28:perf/rust-ael-native-control-lowering

Conversation

@HeshamHM28
Copy link
Copy Markdown
Collaborator

@HeshamHM28 HeshamHM28 commented May 14, 2026

Impact

  • Rust now handles the full verified AEL surface in this repo without real valid-expression fallback: 750/750 valid repo-corpus expressions supported, 596/596 literal parse_ael(...) callsites supported, 0 mismatches.
  • AEL parsing is materially faster:
Corpus Python expr/s Rust expr/s Speedup Python us/expr Rust us/expr
Repo expression corpus 4,798.66 12,988.76 2.71x 208.39 76.99
Literal parse_ael(...) callsites 4,496.22 9,842.60 2.19x 222.41 101.60
  • Real-world parse cost drops by about 121-131 us per dynamically parsed AEL expression.
  • Live RU,50 benchmark stays roughly flat, which is expected: this change speeds up client-side parsing, not Aerospike network/server I/O.

Where this helps

This optimization applies to the client-side AEL parse path:

  • direct parse_ael("...") calls
  • .where("...") on async and sync query builders
  • .where("...") on background/update/delete/UDF builders that accept AEL strings
  • dynamic filters built per request from user input or API parameters

The biggest newly-native cases in this PR are:

  • let(...)
  • when(...)
  • ${var} references and variable definitions
  • operand casts, bitwise expressions, generic functions, and the broader CDT-path coverage already added on this branch

Summary

This PR extends the optional Rust AEL fast path so let(...), when(...), variable references, and variable definitions are lowered natively instead of bouncing back through the Python visitor.

It also keeps broader expression shapes on the Rust path, including operand casts, bitwise expressions, generic functions, and the expanded CDT-path coverage already added on this branch.

Validation

  • cargo test --manifest-path rust/ael_rust_ext/Cargo.toml --target-dir /tmp/ael_rust_ext-target: 5 passed
  • pytest tests/unit -q: 1581 passed
  • pytest tests/integration -q: 979 passed, 37 skipped, 2 xfailed

Repo corpus scan details:

  • scanned 868 expressions total
  • raw skips: 118
  • skip classification: 28 placeholder-only cases with no values supplied, 90 expressions Python itself rejects
  • valid Python-parsable expressions still falling back to Python: 0

Benchmarks

AEL parser throughput

AEL parser latency

Note

I also validated a local CI wiring change that builds the Rust extension in the Python 3.12 unit/integration lanes, but I could not include .github/workflows/build-test.yml in the pushed branch because the GitHub token available in this environment lacks workflow scope for fork pushes.

Add a PyO3-backed Rust subset parser for AEL, wire it into parse_ael as an optional fast path with safe fallback to the existing ANTLR parser, add focused regression tests, and include a standalone parser benchmark tool outside benchmarks/.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant