IFC-2275 Add Jinja2 filters for artifact composition

changelog/+artifact-composition.added.md

@@ -0,0 +1 @@
+Add `artifact_content`, `file_object_content`, `from_json`, and `from_yaml` Jinja2 filters for artifact content composition in templates.

changelog/+artifact-composition.changed.md

@@ -0,0 +1 @@
+Replace `FilterDefinition.trusted: bool` with flag-based `ExecutionContext` model (`CORE`, `WORKER`, `LOCAL`) for context-aware template validation. `validate()` now accepts an optional `context` parameter. Backward compatible.

dev/specs/infp-504-artifact-composition/data-model.md

@@ -104,7 +104,19 @@ def __init__(
 - New optional `client` parameter
 - When `client` provided: instantiate `InfrahubFilters`, register `artifact_content` and `file_object_content`
 - Always register `from_json` and `from_yaml` (no client needed)
-- File-based environment must add `enable_async=True` for async filter support
+- File-based environment already has `enable_async=True` (no change needed)
+
+### Jinja2Template.set_client() (new method)
+
+```python
+def set_client(self, client: InfrahubClient) -> None:
+```
+
+**Purpose**: Deferred client injection — allows creating a `Jinja2Template` first and adding the client later. Also supports replacing a previously set client.
+
+- Calls `_register_client_filters(client)` to bind real filter methods
+- If the Jinja2 environment was already created, patches it in place
+- Without calling `set_client()` (and without passing `client` to `__init__`), client-dependent filters raise `JinjaFilterError` with a descriptive message at render time
 
 ### Jinja2Template.validate() (modified signature)
 

dev/specs/infp-504-artifact-composition/spec.md

@@ -93,8 +93,8 @@ The existing single `restricted: bool` parameter on `validate()` is insufficient
 
 ### Functional requirements
 
-- **FR-001**: `Jinja2Template.__init__` MUST accept an optional `client` parameter of type `InfrahubClient | None` (default `None`). `InfrahubClientSync` is not supported.
-- **FR-002**: A dedicated class (for example, `InfrahubFilters`) MUST be introduced to hold the client reference and expose the Infrahub-specific filter callable methods. `Jinja2Template` instantiates this class when a client is provided and registers its filters into the Jinja2 environment.
+- **FR-001**: `Jinja2Template.__init__` MUST accept an optional `client` parameter of type `InfrahubClient | None` (default `None`). Additionally, `Jinja2Template` MUST expose a `set_client(client)` method for deferred client injection, allowing the template to be created first and the client added later. `InfrahubClientSync` is not supported.
+- **FR-002**: A dedicated class (for example, `InfrahubFilters`) MUST be introduced to hold the client reference and expose the Infrahub-specific filter callable methods. `Jinja2Template` instantiates this class when a client is provided (via `__init__` or `set_client()`) and registers its filters into the Jinja2 environment.
 - **FR-003**: The system MUST provide an `artifact_content` Jinja2 filter that accepts a `storage_id` string and returns the raw string content of the referenced artifact, using the artifact-specific API path.
 - **FR-004**: The system MUST provide a `file_object_content` Jinja2 filter that accepts a `storage_id` string and returns the raw string content of the referenced file object, using the file-object-specific API path or metadata handling — this implementation is distinct from `artifact_content`.
 - **FR-005**: Both `artifact_content` and `file_object_content` MUST raise `JinjaFilterError` when the input `storage_id` is null or empty, or when the object store cannot retrieve the content for any reason (not found, network failure, auth failure). Additionally, `file_object_content` MUST raise `JinjaFilterError` when the retrieved content has a non-text content type (i.e., not `text/*`, `application/json`, or `application/yaml`).

dev/specs/infp-504-artifact-composition/tasks.md

@@ -22,8 +22,8 @@
 
 **CRITICAL**: No user story work can begin until this phase is complete
 
-- [ ] T001 [P] Create `JinjaFilterError` exception class in `infrahub_sdk/template/exceptions.py` — subclass of `JinjaTemplateError` with `filter_name`, `message`, and optional `hint` attributes. Include unit tests for instantiation, inheritance chain, and message formatting. (IFC-2367)
-- [ ] T002 [P] Implement `ExecutionContext` flag enum and migrate `FilterDefinition` in `infrahub_sdk/template/filters.py` — add `ExecutionContext(Flag)` with `CORE`, `WORKER`, `LOCAL`, `ALL` values. Replace `FilterDefinition.trusted: bool` with `allowed_contexts: ExecutionContext`. Add backward-compat `trusted` property. Migrate all 138 existing filter entries (`trusted=True` → `ALL`, `trusted=False` → `LOCAL`). Update `validate()` in `infrahub_sdk/template/__init__.py` to accept optional `context: ExecutionContext` parameter (takes precedence over `restricted`; `restricted=True` → `CORE`, `restricted=False` → `LOCAL`). Include unit tests for all 3 contexts with existing filters, backward compat path, and no regressions. (IFC-2368)
+- [x] T001 [P] Create `JinjaFilterError` exception class in `infrahub_sdk/template/exceptions.py` — subclass of `JinjaTemplateError` with `filter_name`, `message`, and optional `hint` attributes. Include unit tests for instantiation, inheritance chain, and message formatting. (IFC-2367)
+- [x] T002 [P] Implement `ExecutionContext` flag enum and migrate `FilterDefinition` in `infrahub_sdk/template/filters.py` — add `ExecutionContext(Flag)` with `CORE`, `WORKER`, `LOCAL`, `ALL` values. Replace `FilterDefinition.trusted: bool` with `allowed_contexts: ExecutionContext`. Add backward-compat `trusted` property. Migrate all 138 existing filter entries (`trusted=True` → `ALL`, `trusted=False` → `LOCAL`). Update `validate()` in `infrahub_sdk/template/__init__.py` to accept optional `context: ExecutionContext` parameter (takes precedence over `restricted`; `restricted=True` → `CORE`, `restricted=False` → `LOCAL`). Include unit tests for all 3 contexts with existing filters, backward compat path, and no regressions. (IFC-2368)
 
 **Checkpoint**: Foundation ready — JinjaFilterError and ExecutionContext available for all stories
 
@@ -37,9 +37,9 @@
 
 ### Implementation for US1 + US4
 
-- [ ] T003 [US1] Create `InfrahubFilters` class in `infrahub_sdk/template/infrahub_filters.py` — new file. Class holds `InfrahubClient` reference, exposes async filter methods. Methods are `async def` (Jinja2 `auto_await` handles them in async render mode per R-001). Raises `JinjaFilterError` when called without a client. Include unit tests for instantiation with/without client. (IFC-2371)
-- [ ] T004 [US1] Implement `artifact_content` async method on `InfrahubFilters` in `infrahub_sdk/template/infrahub_filters.py` — uses `self.client.object_store.get(identifier=storage_id)`. Raises `JinjaFilterError` on: null/empty storage_id, retrieval failure, permission denied (catch `AuthenticationError` per R-006). Artifacts are always text (no binary check needed per R-003). Include unit tests: happy path (mocked ObjectStore), null, empty, not-found, network error, permission denied, no-client error with descriptive message. (IFC-2372)
-- [ ] T005 [US1] [US4] Add `client` parameter to `Jinja2Template.__init__` and wire up filter registration in `infrahub_sdk/template/__init__.py` — add `client: InfrahubClient | None = None` param. When client provided: instantiate `InfrahubFilters`, register `artifact_content` into Jinja2 env filter map. Add `enable_async=True` to `_get_file_based_environment()` (per R-001 caveat). Register `artifact_content` in `FilterDefinition` registry with `allowed_contexts=ExecutionContext.WORKER`. Include unit tests: render with client (mocked), render without client (error), validation in CORE (blocked), WORKER (allowed), LOCAL (allowed). Verify existing untrusted filters like `safe` remain blocked in WORKER context (US4 AC3). (IFC-2375 partial + IFC-2376 partial)
+- [x] T003 [US1] Create `InfrahubFilters` class in `infrahub_sdk/template/infrahub_filters.py` — new file. Class holds `InfrahubClient` reference, exposes async filter methods. Methods are `async def` (Jinja2 `auto_await` handles them in async render mode per R-001). Raises `JinjaFilterError` when called without a client. Include unit tests for instantiation with/without client. (IFC-2371)
+- [x] T004 [US1] Implement `artifact_content` async method on `InfrahubFilters` in `infrahub_sdk/template/infrahub_filters.py` — uses `self.client.object_store.get(identifier=storage_id)`. Raises `JinjaFilterError` on: null/empty storage_id, retrieval failure, permission denied (catch `AuthenticationError` per R-006). Artifacts are always text (no binary check needed per R-003). Include unit tests: happy path (mocked ObjectStore), null, empty, not-found, network error, permission denied, no-client error with descriptive message. (IFC-2372)
+- [x] T005 [US1] [US4] Add `client` parameter to `Jinja2Template.__init__` and wire up filter registration in `infrahub_sdk/template/__init__.py` — add `client: InfrahubClient | None = None` param. When client provided: instantiate `InfrahubFilters`, register `artifact_content` into Jinja2 env filter map. Add `enable_async=True` to `_get_file_based_environment()` (per R-001 caveat). Register `artifact_content` in `FilterDefinition` registry with `allowed_contexts=ExecutionContext.WORKER`. Include unit tests: render with client (mocked), render without client (error), validation in CORE (blocked), WORKER (allowed), LOCAL (allowed). Verify existing untrusted filters like `safe` remain blocked in WORKER context (US4 AC3). Also added `set_client()` setter for deferred client injection per PR #885 feedback. (IFC-2375 partial + IFC-2376 partial)
 
 **Checkpoint**: US1 + US4 fully functional. `artifact_content` renders in WORKER context, blocked in CORE. MVP complete.
 
@@ -53,9 +53,9 @@
 
 ### Implementation for US2
 
-- [ ] T006 [P] [US2] Add `get_file_by_storage_id()` method to `ObjectStore` in `infrahub_sdk/object_store.py` — async method using endpoint `GET /api/files/by-storage-id/{storage_id}`. Check `content-type` response header: allow `text/*`, `application/json`, `application/yaml`, `application/x-yaml`; reject all others with descriptive error. Handle 401/403 as `AuthenticationError`. Include unit tests: text response, binary rejection, 404, auth failure, network error. (IFC-2373)
-- [ ] T007 [US2] Implement `file_object_content` async method on `InfrahubFilters` in `infrahub_sdk/template/infrahub_filters.py` — uses new `self.client.object_store.get_file_by_storage_id(storage_id)`. Same error handling as `artifact_content` plus binary content error (delegated to ObjectStore). Include unit tests: happy path, all error conditions, binary content rejection. (IFC-2374)
-- [ ] T008 [US2] Register `file_object_content` filter in `Jinja2Template` and `FilterDefinition` in `infrahub_sdk/template/__init__.py` and `infrahub_sdk/template/filters.py` — register when client provided. `allowed_contexts=ExecutionContext.WORKER`. Include unit tests: render with client, validation in CORE (blocked), WORKER (allowed). (IFC-2375 partial + IFC-2376 partial)
+- [x] T006 [P] [US2] Add `get_file_by_storage_id()` method to `ObjectStore` in `infrahub_sdk/object_store.py` — async method using endpoint `GET /api/files/by-storage-id/{storage_id}`. Check `content-type` response header: allow `text/*`, `application/json`, `application/yaml`, `application/x-yaml`; reject all others with descriptive error. Handle 401/403 as `AuthenticationError`. Include unit tests: text response, binary rejection, 404, auth failure, network error. (IFC-2373)
+- [x] T007 [US2] Implement `file_object_content` async method on `InfrahubFilters` in `infrahub_sdk/template/infrahub_filters.py` — uses new `self.client.object_store.get_file_by_storage_id(storage_id)`. Same error handling as `artifact_content` plus binary content error (delegated to ObjectStore). Include unit tests: happy path, all error conditions, binary content rejection. (IFC-2374)
+- [x] T008 [US2] Register `file_object_content` filter in `Jinja2Template` and `FilterDefinition` in `infrahub_sdk/template/__init__.py` and `infrahub_sdk/template/filters.py` — register when client provided. `allowed_contexts=ExecutionContext.WORKER`. Include unit tests: render with client, validation in CORE (blocked), WORKER (allowed). (IFC-2375 partial + IFC-2376 partial)
 
 **Checkpoint**: US2 complete. `file_object_content` works alongside `artifact_content`.
 
@@ -69,9 +69,9 @@
 
 ### Implementation for US3
 
-- [ ] T009 [P] [US3] Implement `from_json` filter function in `infrahub_sdk/template/infrahub_filters.py` — pure sync function (no client needed). Empty string → `{}` (explicit special-case since `json.loads("")` raises). Malformed JSON → `JinjaFilterError`. Register in `FilterDefinition` with `allowed_contexts=ExecutionContext.ALL`. Register unconditionally in `Jinja2Template._set_filters()`. Include unit tests: valid JSON dict, valid JSON list, empty string → `{}`, malformed → error, render through template. (IFC-2369)
-- [ ] T010 [P] [US3] Implement `from_yaml` filter function in `infrahub_sdk/template/infrahub_filters.py` — pure sync function. Empty string → `{}` (explicit special-case since `yaml.safe_load("")` returns `None`). Malformed YAML → `JinjaFilterError`. Register in `FilterDefinition` with `allowed_contexts=ExecutionContext.ALL`. Register unconditionally in `Jinja2Template._set_filters()`. Include unit tests: valid YAML mapping, valid YAML list, empty string → `{}`, malformed → error, render through template. (IFC-2370)
-- [ ] T011 [US3] Integration test for filter chaining in `tests/unit/template/test_infrahub_filters.py` — test `artifact_content | from_json` and `artifact_content | from_yaml` end-to-end with mocked ObjectStore returning JSON/YAML content. Verify template can access parsed fields. (IFC-2376 partial, SC-006)
+- [x] T009 [P] [US3] Implement `from_json` filter function in `infrahub_sdk/template/infrahub_filters.py` — pure sync function (no client needed). Empty string → `{}` (explicit special-case since `json.loads("")` raises). Malformed JSON → `JinjaFilterError`. Register in `FilterDefinition` with `allowed_contexts=ExecutionContext.ALL`. Register unconditionally in `Jinja2Template._set_filters()`. Include unit tests: valid JSON dict, valid JSON list, empty string → `{}`, malformed → error, render through template. (IFC-2369)
+- [x] T010 [P] [US3] Implement `from_yaml` filter function in `infrahub_sdk/template/infrahub_filters.py` — pure sync function. Empty string → `{}` (explicit special-case since `yaml.safe_load("")` returns `None`). Malformed YAML → `JinjaFilterError`. Register in `FilterDefinition` with `allowed_contexts=ExecutionContext.ALL`. Register unconditionally in `Jinja2Template._set_filters()`. Include unit tests: valid YAML mapping, valid YAML list, empty string → `{}`, malformed → error, render through template. (IFC-2370)
+- [x] T011 [US3] Integration test for filter chaining in `tests/unit/template/test_infrahub_filters.py` — test `artifact_content | from_json` and `artifact_content | from_yaml` end-to-end with mocked ObjectStore returning JSON/YAML content. Verify template can access parsed fields. (IFC-2376 partial, SC-006)
 
 **Checkpoint**: US3 complete. All 4 filters work, chain correctly, and are validated per context.
 

infrahub_sdk/object_store.py

@@ -11,6 +11,14 @@
     from .client import InfrahubClient, InfrahubClientSync
 
 
+ALLOWED_TEXT_CONTENT_TYPES = {"application/json", "application/yaml", "application/x-yaml"}
+
+
+def _extract_content_type(response: httpx.Response) -> str:
+    """Extract and normalize the content-type from an HTTP response, stripping parameters."""
+    return response.headers.get("content-type", "").split(";")[0].strip().lower()
+
+
 class ObjectStoreBase:
     pass
 
@@ -62,6 +70,38 @@ async def upload(self, content: str, tracker: str | None = None) -> dict[str, st
 
         return resp.json()
 
+    async def get_file_by_storage_id(self, storage_id: str, tracker: str | None = None) -> str:
+        """Retrieve file object content by storage_id.
+
+        Raises an error if the content-type indicates binary content.
+        """
+        url = f"{self.client.address}/api/files/by-storage-id/{storage_id}"
+        headers = copy.copy(self.client.headers or {})
+        if self.client.insert_tracker and tracker:
+            headers["X-Infrahub-Tracker"] = tracker
+
+        try:
+            resp = await self.client._get(url=url, headers=headers)
+            resp.raise_for_status()
+        except ServerNotReachableError:
+            self.client.log.error(f"Unable to connect to {self.client.address} .. ")
+            raise
+        except httpx.HTTPStatusError as exc:
+            if exc.response.status_code in {401, 403}:
+                response = exc.response.json()
+                errors = response.get("errors")
+                messages = [error.get("message") for error in errors]
+                raise AuthenticationError(" | ".join(messages)) from exc
+            raise
+
+        content_type = _extract_content_type(resp)
+        if not content_type.startswith("text/") and content_type not in ALLOWED_TEXT_CONTENT_TYPES:
+            raise ValueError(
+                f"Binary content not supported: content-type '{content_type}' for storage_id '{storage_id}'"
+            )
+
+        return resp.text
+
 
 class ObjectStoreSync(ObjectStoreBase):
     def __init__(self, client: InfrahubClientSync) -> None:
@@ -109,3 +149,35 @@ def upload(self, content: str, tracker: str | None = None) -> dict[str, str]:
                 raise AuthenticationError(" | ".join(messages)) from exc
 
         return resp.json()
+
+    def get_file_by_storage_id(self, storage_id: str, tracker: str | None = None) -> str:
+        """Retrieve file object content by storage_id.
+
+        Raises an error if the content-type indicates binary content.
+        """
+        url = f"{self.client.address}/api/files/by-storage-id/{storage_id}"
+        headers = copy.copy(self.client.headers or {})
+        if self.client.insert_tracker and tracker:
+            headers["X-Infrahub-Tracker"] = tracker
+
+        try:
+            resp = self.client._get(url=url, headers=headers)
+            resp.raise_for_status()
+        except ServerNotReachableError:
+            self.client.log.error(f"Unable to connect to {self.client.address} .. ")
+            raise
+        except httpx.HTTPStatusError as exc:
+            if exc.response.status_code in {401, 403}:
+                response = exc.response.json()
+                errors = response.get("errors")
+                messages = [error.get("message") for error in errors]
+                raise AuthenticationError(" | ".join(messages)) from exc
+            raise
+
+        content_type = _extract_content_type(resp)
+        if not content_type.startswith("text/") and content_type not in ALLOWED_TEXT_CONTENT_TYPES:
+            raise ValueError(
+                f"Binary content not supported: content-type '{content_type}' for storage_id '{storage_id}'"
+            )
+
+        return resp.text