feat(lapis): also allow multiple value in int, float and date filters (and concatenate them with or)#1626
Conversation
…concatenate them with `or`)
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
or)or)
or)or)
There was a problem hiding this comment.
Pull request overview
Adds multi-value exact-match support (logical OR semantics) for int, float, and date metadata filters in LAPIS—bringing them in line with existing multi-value behavior for string filters—and updates docs/tests accordingly.
Changes:
- Extend
int/float/dateexact-match filters to accept multiple values and map them to anOr(...)SILO filter expression (includingnullas “is missing”). - Update OpenAPI + template/docs + docs-site filter descriptions to reflect new multi-value capability.
- Add/adjust unit + e2e coverage for multi-value exact-match queries.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
lapis/src/main/kotlin/org/genspectrum/lapis/model/SiloFilterExpressionMapper.kt |
Implements OR-combination for multi-value exact-match int/float/date filters (including null). |
lapis/src/main/kotlin/org/genspectrum/lapis/openApi/OpenApiDocs.kt |
Updates filter schemas so int/float/date exact-match filters accept either a scalar or an array. |
lapis/src/main/resources/templates/llms.txt |
Updates end-user docs to describe array support for date/int/float exact matches. |
lapis/src/test/kotlin/org/genspectrum/lapis/model/SiloFilterExpressionMapperTest.kt |
Adjusts/extends unit test expectations for OR-wrapped exact-match expressions and new multi-value cases. |
lapis-e2e/test/aggregatedQueries/intEqualsMultiple.json |
New e2e aggregated-query fixture validating multi-value int equals behavior. |
lapis-e2e/test/aggregatedQueries/floatEqualsMultiple.json |
New e2e aggregated-query fixture validating multi-value float equals behavior. |
lapis-e2e/test/aggregatedQueries/dateEqualsMultiple.json |
New e2e aggregated-query fixture validating multi-value date equals behavior. |
lapis-e2e/test/aminoAcidSequence.spec.ts |
Updates an assertion to match a changed SILO error message wording. |
lapis-docs/src/components/FiltersTable/getFilters.tsx |
Updates docs-site filter descriptions to mention array/OR semantics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| description: | ||
| `Filters the "${metadata.name}" column with exact match. ` + | ||
| `You can also supply an array of values - they will be combined with logical OR.`, |
There was a problem hiding this comment.
The filter docs generation for string fields still contains an extra double quote in column" with exact match, which results in a typo/inconsistent wording compared to the int/float/date descriptions. Remove the stray quote so the sentence reads correctly.
| SequenceFilterFieldType.Int -> { | ||
| val fieldSchema = Schema<Int>().types(setOf(fieldType.openApiType)) | ||
| Schema<Any>().anyOf( | ||
| listOf( | ||
| fieldSchema, | ||
| logicalOrArraySchema(fieldSchema), | ||
| ), | ||
| ) |
There was a problem hiding this comment.
The new OpenAPI schema allows int/float/date filters to be either a single value or an array, but it does not document that null is also accepted as an array element (and as a single value) to filter for missing values. Consider making the schema (and array item schema) nullable (or using an anyOf that includes null) so the OpenAPI contract matches the implemented behavior.
| [# th:if="${dateField != null}"]- **Date fields**: Use exact match, or `From`/`To` suffixes for ranges. You can also supply an array for exact match - the values will be combined with logical OR. | ||
| Examples: `"[(${dateField})]": "2023-01-15"`, `"[(${dateField})]From": "2023-01-01", "[(${dateField})]To": "2023-12-31"`, `"[(${dateField})]": ["2023-01-15", "2023-02-20"]`[/] | ||
| [# th:if="${intField != null}"]- **Integer fields**: Use exact match or `From`/`To` for ranges. You can also supply an array for exact match - the values will be combined with logical OR. | ||
| Examples: `"[(${intField})]": 42`, `"[(${intField})]From": 10, "[(${intField})]To": 50`, `"[(${intField})]": [42, 43]`[/] | ||
| [# th:if="${floatField != null}"]- **Float fields**: Use exact match or `From`/`To` for ranges. You can also supply an array for exact match - the values will be combined with logical OR. |
There was a problem hiding this comment.
This section documents array support for exact date/int/float matches, but it doesn’t mention that null can be included as an array element to match missing values (as described in the PR). Add a short note and/or an example including null (or explicitly point users to the existing .isNull filter syntax) to avoid conflicting guidance.
| [# th:if="${dateField != null}"]- **Date fields**: Use exact match, or `From`/`To` suffixes for ranges. You can also supply an array for exact match - the values will be combined with logical OR. | |
| Examples: `"[(${dateField})]": "2023-01-15"`, `"[(${dateField})]From": "2023-01-01", "[(${dateField})]To": "2023-12-31"`, `"[(${dateField})]": ["2023-01-15", "2023-02-20"]`[/] | |
| [# th:if="${intField != null}"]- **Integer fields**: Use exact match or `From`/`To` for ranges. You can also supply an array for exact match - the values will be combined with logical OR. | |
| Examples: `"[(${intField})]": 42`, `"[(${intField})]From": 10, "[(${intField})]To": 50`, `"[(${intField})]": [42, 43]`[/] | |
| [# th:if="${floatField != null}"]- **Float fields**: Use exact match or `From`/`To` for ranges. You can also supply an array for exact match - the values will be combined with logical OR. | |
| [# th:if="${dateField != null}"]- **Date fields**: Use exact match, or `From`/`To` suffixes for ranges. You can also supply an array for exact match - the values will be combined with logical OR, and may include `null` to match missing values. | |
| Examples: `"[(${dateField})]": "2023-01-15"`, `"[(${dateField})]From": "2023-01-01", "[(${dateField})]To": "2023-12-31"`, `"[(${dateField})]": ["2023-01-15", "2023-02-20"]`[/] | |
| [# th:if="${intField != null}"]- **Integer fields**: Use exact match or `From`/`To` for ranges. You can also supply an array for exact match - the values will be combined with logical OR, and may include `null` to match missing values. | |
| Examples: `"[(${intField})]": 42`, `"[(${intField})]From": 10, "[(${intField})]To": 50`, `"[(${intField})]": [42, 43]`[/] | |
| [# th:if="${floatField != null}"]- **Float fields**: Use exact match or `From`/`To` for ranges. You can also supply an array for exact match - the values will be combined with logical OR, and may include `null` to match missing values. |
| const errorResponse = await response.json(); | ||
| expect(errorResponse.error.detail).to.match( | ||
| /Error from SILO: The table does not contain the SequenceColumn 'unknownGene'/ | ||
| /Error from SILO: The table does not contain the field unknownGene/ |
There was a problem hiding this comment.
Unrelated to this PR, but SILO recently changed the error message in this case.
resolves #1595
int,float, anddateexact-match filters now accept an array of values, combining them with logical OR — the same behaviour already supported forstringfields.{ "age": [50, 51] } { "qcValue": [0.95, 0.97] } { "date": ["2021-05-08", "2021-01-20"] }nullis supported as an array element to filter for missing values. Range filters (*From/*To) are unchanged and still accept a single value only.PR Checklist
llms.txt.