Skip to content

IFC-2275 Add Jinja2 filters for artifact composition#889

Open
gmazoyer wants to merge 10 commits intogma-20260323-infp504-planfrom
gma/infp-504-artifact-composition
Open

IFC-2275 Add Jinja2 filters for artifact composition#889
gmazoyer wants to merge 10 commits intogma-20260323-infp504-planfrom
gma/infp-504-artifact-composition

Conversation

@gmazoyer
Copy link
Contributor

@gmazoyer gmazoyer commented Mar 24, 2026

Why

Customers building modular configuration pipelines need to compose larger artifacts from smaller sub-artifacts inside Jinja2 transforms, without duplicating template logic or GraphQL query fields. Today there is no way for a template to reference the rendered content of another artifact.

This PR implements the SDK side of the "Artifact of Artifacts" initiative, adding new Jinja2 filters and the supporting infrastructure to enable artifact content composition.

What changed

  • artifact_content that accepts a storage_id string, fetches artifact content from the object store via ObjectStore.get()
  • file_object_content with the same pattern but uses the /api/files/by-storage-id/{storage_id} endpoint, with binary content-type rejection
  • from_json / from_yaml to parse strings into Python dicts/lists, enabling chaining like {{ sid | artifact_content | from_json }}
  • Replaced FilterDefinition.trusted: bool with allowed_contexts: ExecutionContext using Python's Flag enum
  • Three contexts: CORE (API server computed attributes, most restrictive), WORKER (Prefect workers), LOCAL (CLI/unrestricted)
  • New optional client: InfrahubClient constructor parameter
  • New set_client() method for deferred client injection (per PR Add plan to implement INFP-504 #885 feedback)
  • Without a client, client-dependent filters raise JinjaFilterError with an actionable message
  • JinjaFilterError(filter_name, message, hint) subclassing JinjaTemplateError
  • New get_file_by_storage_id() method (async + sync) using /api/files/by-storage-id/{storage_id}
  • Content-type validation: rejects non-text responses (application/octet-stream, etc.)

Suggested review order

  1. infrahub_sdk/template/filters.py for ExecutionContext enum and FilterDefinition migration
  2. infrahub_sdk/template/infrahub_filters.py for InfrahubFilters class, from_json, from_yaml
    3 infrahub_sdk/template/__init__.py for Jinja2Template wiring, set_client(), validate() changes
  3. infrahub_sdk/object_store.py for get_file_by_storage_id()

How to review

  • validate() context logic iit is the core of the trust model change
  • _register_client_filters() and set_client() ensure client-dependent filters are correctly bound or fallback

How to test

uv run pytest tests/unit/sdk/test_infrahub_filters.py -v
uv run pytest tests/unit/sdk/test_template.py -v
uv run pytest tests/unit/sdk/test_object_store.py -v

Impact & rollout

  • Backward compatibility: validate(restricted=True/False) continues to work as before. FilterDefinition.trusted property preserved. Jinja2Template constructor is backward compatible (new client param is optional with default None).
  • Server-side change: required but it is a part of this body of work

Checklist

  • Tests added/updated
  • Changelog entry added (uv run towncrier create ...)
  • External docs updated (if user-facing or ops-facing change)
  • Internal .md docs updated (internal knowledge and AI code tools knowledge)

@coderabbitai
Copy link

coderabbitai bot commented Mar 24, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (4)
  • main
  • stable
  • develop
  • release-.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3f736b96-f3d0-421c-acbd-ce9d2895f62e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 24, 2026

Deploying infrahub-sdk-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: 02b33a1
Status: ✅  Deploy successful!
Preview URL: https://e43d48d6.infrahub-sdk-python.pages.dev
Branch Preview URL: https://gma-infp-504-artifact-compos.infrahub-sdk-python.pages.dev

View logs

@gmazoyer gmazoyer force-pushed the gma/infp-504-artifact-composition branch from ce7a42e to d7514e6 Compare March 24, 2026 08:47
@codecov
Copy link

codecov bot commented Mar 24, 2026

Codecov Report

❌ Patch coverage is 79.60526% with 31 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
infrahub_sdk/object_store.py 46.80% 23 Missing and 2 partials ⚠️
infrahub_sdk/template/__init__.py 86.66% 1 Missing and 3 partials ⚠️
infrahub_sdk/template/infrahub_filters.py 96.36% 1 Missing and 1 partial ⚠️
@@                     Coverage Diff                      @@
##             gma-20260323-infp504-plan     #889   +/-   ##
============================================================
  Coverage                             ?   80.85%           
============================================================
  Files                                ?      120           
  Lines                                ?    10454           
  Branches                             ?     1557           
============================================================
  Hits                                 ?     8453           
  Misses                               ?     1485           
  Partials                             ?      516           
Flag Coverage Δ
integration-tests 39.66% <0.00%> (?)
python-3.10 52.28% <54.60%> (?)
python-3.11 52.28% <54.60%> (?)
python-3.12 52.28% <54.60%> (?)
python-3.13 52.30% <54.60%> (?)
python-3.14 53.98% <56.29%> (?)
python-filler-3.12 23.95% <31.57%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
infrahub_sdk/template/exceptions.py 96.96% <100.00%> (ø)
infrahub_sdk/template/filters.py 100.00% <100.00%> (ø)
infrahub_sdk/template/infrahub_filters.py 96.36% <96.36%> (ø)
infrahub_sdk/template/__init__.py 94.70% <86.66%> (ø)
infrahub_sdk/object_store.py 59.09% <46.80%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gmazoyer gmazoyer marked this pull request as ready for review March 24, 2026 10:45
@gmazoyer gmazoyer requested a review from a team as a code owner March 24, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant