Skip to content

feat: support concat(toArray(...)) scalar includes#1384

Open
KyleAMathews wants to merge 2 commits intomainfrom
concat-to-array-query
Open

feat: support concat(toArray(...)) scalar includes#1384
KyleAMathews wants to merge 2 commits intomainfrom
concat-to-array-query

Conversation

@KyleAMathews
Copy link
Collaborator

Summary

  • add scalar includes materialization so toArray(subquery.select(({ c }) => c.text)) produces scalar arrays
  • support concat(toArray(...)) by lowering it through includes materialization instead of introducing a new aggregate
  • make scalar child select() type-check without user-facing casts while preserving existing object-select inference

Design

  • extend includes materialization from collection to collection | array | concat
  • wrap scalar child selects in an internal __includes_scalar__ field so the existing includes pipeline can keep using object-shaped child rows
  • materialize inline array/concat values in the live query output layer and re-emit parent rows when child chunks change
  • add builder/type support for scalar select() results without breaking spread/object projections

Benchmarks

Actual live-query benchmark for:

  • content: concat(toArray(q.from({ c: chunkCollection }).where(...).orderBy(...).select(({ c }) => c.text)))
1 message x 2000 chunks
concatToArray  preload 17.52ms  append 0.94ms  mid update 0.95ms
stringAgg      preload 13.21ms  append 1.29ms  mid update 1.53ms

100 messages x 100 chunks
concatToArray  preload 70.55ms  append 1.14ms  mid update 1.06ms
stringAgg      preload 65.46ms  append 0.88ms  mid update 1.00ms

Update performance is competitive, and the single-message incremental case is slightly better on the concat/toArray path. Preload is still a bit slower than the purpose-built aggregate.

Verification

pnpm exec tsc -p packages/db/tsconfig.json --noEmit --pretty false
pnpm vitest --run packages/db/tests/query/builder/functions.test.ts packages/db/tests/query/includes.test.ts packages/db/tests/query/group-by.test.ts

Notes

  • This replaces the design from feat: add stringAgg aggregate function #1382 rather than extending groupBy with a new aggregate.
  • The separate .select(...).fn.select(...) idea is still blocked because fn.select runs before includes materialize.

@pkg-pr-new
Copy link

pkg-pr-new bot commented Mar 18, 2026

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/angular-db@1384

@tanstack/db

npm i https://pkg.pr.new/TanStack/db/@tanstack/db@1384

@tanstack/db-browser-wa-sqlite-persisted-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-browser-wa-sqlite-persisted-collection@1384

@tanstack/db-electron-sqlite-persisted-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-electron-sqlite-persisted-collection@1384

@tanstack/db-ivm

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-ivm@1384

@tanstack/db-node-sqlite-persisted-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-node-sqlite-persisted-collection@1384

@tanstack/db-react-native-sqlite-persisted-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-react-native-sqlite-persisted-collection@1384

@tanstack/db-sqlite-persisted-collection-core

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-sqlite-persisted-collection-core@1384

@tanstack/electric-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/electric-db-collection@1384

@tanstack/offline-transactions

npm i https://pkg.pr.new/TanStack/db/@tanstack/offline-transactions@1384

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/powersync-db-collection@1384

@tanstack/query-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/query-db-collection@1384

@tanstack/react-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/react-db@1384

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/rxdb-db-collection@1384

@tanstack/solid-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/solid-db@1384

@tanstack/svelte-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/svelte-db@1384

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/trailbase-db-collection@1384

@tanstack/vue-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/vue-db@1384

commit: 64ba7ab

@github-actions
Copy link
Contributor

Size Change: +355 B (+0.32%)

Total Size: 111 kB

Filename Size Change
./packages/db/dist/esm/query/builder/functions.js 922 B +130 B (+16.41%) ⚠️
./packages/db/dist/esm/query/builder/index.js 5.2 kB +47 B (+0.91%)
./packages/db/dist/esm/query/compiler/index.js 3.63 kB +5 B (+0.14%)
./packages/db/dist/esm/query/ir.js 829 B +45 B (+5.74%) 🔍
./packages/db/dist/esm/query/live/collection-config-builder.js 7.76 kB +128 B (+1.68%)
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.39 kB
./packages/db/dist/esm/collection/changes.js 1.38 kB
./packages/db/dist/esm/collection/cleanup-queue.js 810 B
./packages/db/dist/esm/collection/events.js 434 B
./packages/db/dist/esm/collection/index.js 3.69 kB
./packages/db/dist/esm/collection/indexes.js 2.35 kB
./packages/db/dist/esm/collection/lifecycle.js 1.76 kB
./packages/db/dist/esm/collection/mutations.js 2.47 kB
./packages/db/dist/esm/collection/state.js 5.2 kB
./packages/db/dist/esm/collection/subscription.js 3.71 kB
./packages/db/dist/esm/collection/sync.js 2.43 kB
./packages/db/dist/esm/collection/transaction-metadata.js 144 B
./packages/db/dist/esm/deferred.js 207 B
./packages/db/dist/esm/errors.js 4.83 kB
./packages/db/dist/esm/event-emitter.js 748 B
./packages/db/dist/esm/index.js 2.85 kB
./packages/db/dist/esm/indexes/auto-index.js 777 B
./packages/db/dist/esm/indexes/base-index.js 766 B
./packages/db/dist/esm/indexes/btree-index.js 2.17 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.24 kB
./packages/db/dist/esm/indexes/reverse-index.js 538 B
./packages/db/dist/esm/local-only.js 890 B
./packages/db/dist/esm/local-storage.js 2.1 kB
./packages/db/dist/esm/optimistic-action.js 359 B
./packages/db/dist/esm/paced-mutations.js 496 B
./packages/db/dist/esm/proxy.js 3.75 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 1.05 kB
./packages/db/dist/esm/query/compiler/evaluators.js 1.62 kB
./packages/db/dist/esm/query/compiler/expressions.js 430 B
./packages/db/dist/esm/query/compiler/group-by.js 2.69 kB
./packages/db/dist/esm/query/compiler/joins.js 2.11 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.5 kB
./packages/db/dist/esm/query/compiler/select.js 1.11 kB
./packages/db/dist/esm/query/effect.js 4.78 kB
./packages/db/dist/esm/query/expression-helpers.js 1.43 kB
./packages/db/dist/esm/query/live-query-collection.js 360 B
./packages/db/dist/esm/query/live/collection-registry.js 264 B
./packages/db/dist/esm/query/live/collection-subscriber.js 1.94 kB
./packages/db/dist/esm/query/live/internal.js 145 B
./packages/db/dist/esm/query/live/utils.js 1.57 kB
./packages/db/dist/esm/query/optimizer.js 2.62 kB
./packages/db/dist/esm/query/predicate-utils.js 2.97 kB
./packages/db/dist/esm/query/query-once.js 359 B
./packages/db/dist/esm/query/subset-dedupe.js 960 B
./packages/db/dist/esm/scheduler.js 1.3 kB
./packages/db/dist/esm/SortedMap.js 1.3 kB
./packages/db/dist/esm/strategies/debounceStrategy.js 247 B
./packages/db/dist/esm/strategies/queueStrategy.js 428 B
./packages/db/dist/esm/strategies/throttleStrategy.js 246 B
./packages/db/dist/esm/transactions.js 2.9 kB
./packages/db/dist/esm/utils.js 927 B
./packages/db/dist/esm/utils/browser-polyfills.js 304 B
./packages/db/dist/esm/utils/btree.js 5.61 kB
./packages/db/dist/esm/utils/comparison.js 1.05 kB
./packages/db/dist/esm/utils/cursor.js 457 B
./packages/db/dist/esm/utils/index-optimization.js 1.54 kB
./packages/db/dist/esm/utils/type-guards.js 157 B
./packages/db/dist/esm/virtual-props.js 360 B

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

Size Change: 0 B

Total Size: 4.23 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 249 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.32 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.34 kB
./packages/react-db/dist/esm/useLiveQueryEffect.js 355 B
./packages/react-db/dist/esm/useLiveSuspenseQuery.js 559 B
./packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant