Skip to content

feat: replace container base images with pixi-managed environments#810

Merged
fstagni merged 7 commits intoDIRACGrid:mainfrom
chrisburr:feat/pixi-containers
Mar 26, 2026
Merged

feat: replace container base images with pixi-managed environments#810
fstagni merged 7 commits intoDIRACGrid:mainfrom
chrisburr:feat/pixi-containers

Conversation

@chrisburr
Copy link
Copy Markdown
Member

@chrisburr chrisburr commented Feb 26, 2026

Closes #763

Replace container builds from the multi-repo base image approach
(micromamba + conda environment.yml + pip install --no-deps) with
self-contained, multi-stage Dockerfiles using pixi environments from the
committed lock file. This ensures reproducible builds with exact version
parity between development and production.

Summary

  • New unified containers/Dockerfile that builds both services and
    client images via a PIXI_ENV build arg
  • pixi.lock is now committed (previously gitignored) to pin all
    dependencies
  • Container environments (container-services, container-client) are
    defined in pixi.toml in the same solve group as dev environments
  • Extension (gubbins) Dockerfile follows the same pattern; fetches the
    entrypoint from the matching diracx version at build time to avoid
    maintaining a duplicate
  • linux-aarch64 added to supported platforms
  • CI workflows updated to build and load images for both diracx and
    gubbins integration tests
  • Old per-image Dockerfiles (containers/client/, containers/services/,
    extensions/containers/) removed
  • Companion commits fix an OTel KeyError with sdk >=1.28.0 and pin
    importlib-metadata <8.8.0 to work around a pixi conda/PyPI conflict

See also #852 for the planned split of extensions/gubbins/ into a
standalone repository.

@chrisburr chrisburr force-pushed the feat/pixi-containers branch from 7dbead8 to e840466 Compare February 26, 2026 15:31
@read-the-docs-community
Copy link
Copy Markdown

read-the-docs-community bot commented Feb 26, 2026

Documentation build overview

📚 diracx | 🛠️ Build #31969107 | 📁 Comparing 1a2ef83 against latest (e552651)


🔍 Preview build

Show files changed (2 files in total): 📝 2 modified | ➕ 0 added | ➖ 0 deleted
File Status
dev/explanations/components/index.html 📝 modified
dev/explanations/extensions/index.html 📝 modified

@chrisburr chrisburr force-pushed the feat/pixi-containers branch 13 times, most recently from a86608d to c11d95c Compare March 24, 2026 16:07
@chrisburr chrisburr marked this pull request as ready for review March 24, 2026 16:07
@chaen
Copy link
Copy Markdown
Contributor

chaen commented Mar 24, 2026

I think the problem here is that we need to override the entrypoint of the keystore init job as well, but I can't try right now

ARG PIXI_ENV=container-services
ARG DIRACX_EXTENSIONS=diracx
ENV PIXI_ENV=${PIXI_ENV}
ENV DIRACX_EXTENSIONS=${DIRACX_EXTENSIONS}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You redefine DIRACX_EXTENSIONS from 2 lines above

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the env var not the arg?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case, this variable is no longer needed since DIRACGrid/DIRAC#8488

with:
context: extensions/containers/services
context: .
file: containers/Dockerfile
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should that be extensions/containers/Dockerfile ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it builds from the top level pixi.toml

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK so the extensions/containers/Dockerfile is just for docs purposes ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

chrisburr and others added 4 commits March 25, 2026 09:48
…28.0

Since opentelemetry-sdk v1.28.0 (open-telemetry/opentelemetry-python#4166),
LoggingHandler._translate() calls self.format(record) when a formatter is
set. The existing formatter uses %(otelTraceID)s placeholders but
set_logging_format=False meant the LoggingInstrumentor never injected
those attributes, causing a KeyError at runtime.
@chrisburr chrisburr force-pushed the feat/pixi-containers branch from a8350cf to 9b8f0a6 Compare March 25, 2026 08:50
@chrisburr chrisburr force-pushed the feat/pixi-containers branch from 9b8f0a6 to 71d4284 Compare March 25, 2026 08:58
@chrisburr chrisburr force-pushed the feat/pixi-containers branch from 1002637 to 1a2ef83 Compare March 25, 2026 15:17
@chrisburr
Copy link
Copy Markdown
Member Author

I think there is nothing left blocking this from being merged.

@fstagni fstagni merged commit 7cae3ca into DIRACGrid:main Mar 26, 2026
33 of 35 checks passed
chrisburr added a commit to chrisburr/diracx that referenced this pull request Mar 26, 2026
Container images are now built from source via pixi (since DIRACGrid#810),
so editable pip installs of mounted source dirs are redundant.
Removing them fixes the gubbins init-cs failure caused by git
"dubious ownership" errors when the container UID differs from
the mounted source owner.
chrisburr added a commit to chrisburr/diracx that referenced this pull request Mar 27, 2026
Container images are now built from source via pixi (since DIRACGrid#810),
so editable pip installs of mounted source dirs are redundant.
Removing them fixes the gubbins init-cs failure caused by git
"dubious ownership" errors when the container UID differs from
the mounted source owner.
aldbr pushed a commit to chrisburr/diracx that referenced this pull request Mar 27, 2026
Container images are now built from source via pixi (since DIRACGrid#810),
so editable pip installs of mounted source dirs are redundant.
Removing them fixes the gubbins init-cs failure caused by git
"dubious ownership" errors when the container UID differs from
the mounted source owner.
fstagni pushed a commit that referenced this pull request Mar 27, 2026
Container images are now built from source via pixi (since #810),
so editable pip installs of mounted source dirs are redundant.
Removing them fixes the gubbins init-cs failure caused by git
"dubious ownership" errors when the container UID differs from
the mounted source owner.
chrisburr added a commit to chrisburr/diracx that referenced this pull request Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Design Discussion] Do we still need to rely on the container-images repository?

3 participants