feat: add diracx-tasks by chrisburr · Pull Request #842 · DIRACGrid/diracx

chrisburr · 2026-03-20T15:57:49Z

No description provided.

chaen · 2026-03-21T13:59:51Z

@fstagni the doc does not build in RTD, in the meantime, these are the page worth mentioning to whoever you think is relevant:
https://github.com/chrisburr/diracx/blob/feat/tasks/docs/dev/explanations/tasks/index.md
https://github.com/chrisburr/diracx/blob/feat/tasks/docs/adr/DX-ADR-001_tasks.md

read-the-docs-community · 2026-03-21T22:10:19Z

Documentation build overview

📚 diracx | 🛠️ Build #32002408 | 📁 Comparing 65f2e76 against latest (07c549d)

🔍 Preview build

Show files changed (144 files in total): 📝 117 modified | ➕ 25 added | ➖ 2 deleted

File	Status
404.html	📝 modified
index.html	📝 modified
REFERENCE/index.html	📝 modified
RUN_PROD/index.html	📝 modified
SECURITY/index.html	📝 modified
SSO/index.html	📝 modified
admin/index.html	📝 modified
adr/index.html	➕ added
dev/index.html	📝 modified
developers/index.html	📝 modified
roadmap/index.html	📝 modified
user/index.html	📝 modified
admin/explanations/index.html	📝 modified
admin/how-to/index.html	📝 modified
admin/reference/index.html	📝 modified
admin/tutorials/index.html	📝 modified
adr/DX-ADR-001_tasks/index.html	➕ added
adr/DX-ADR-XXX_template/index.html	➕ added
dev/explanations/index.html	📝 modified
dev/how-to/index.html	📝 modified
dev/reference/index.html	📝 modified
dev/tutorials/index.html	📝 modified
user/explanations/index.html	📝 modified
user/how-to/index.html	📝 modified
user/reference/index.html	📝 modified
user/tutorials/index.html	📝 modified
admin/explanations/auth-with-diracx/index.html	📝 modified
admin/explanations/auth-with-external/index.html	📝 modified
admin/explanations/chart-structure/index.html	📝 modified
admin/explanations/configuration/index.html	📝 modified
admin/explanations/database-management/index.html	📝 modified
admin/explanations/manage-web-release/index.html	📝 modified
admin/explanations/opentelemetry/index.html	📝 modified
admin/explanations/sandbox-store/index.html	📝 modified
admin/explanations/tasks/index.html	➕ added
admin/explanations/user-management/index.html	📝 modified
admin/how-to/debugging/index.html	📝 modified
admin/how-to/install/index.html	📝 modified
admin/how-to/rotate-a-secret/index.html	📝 modified
admin/how-to/tasks/index.html	➕ added
admin/how-to/upgrading/index.html	📝 modified
admin/reference/env-variables/index.html	📝 modified
admin/reference/security_model/index.html	📝 modified
admin/reference/settings-and-preferences/index.html	📝 modified
admin/reference/tasks-configuration/index.html	➕ added
admin/reference/values/index.html	📝 modified
admin/tutorials/authentication/index.html	📝 modified
admin/tutorials/monitor-tasks/index.html	➕ added
admin/tutorials/run_locally/index.html	📝 modified
dev/explanations/components/index.html	📝 modified
dev/explanations/dependency-management/index.html	📝 modified
dev/explanations/designing-functionality/index.html	📝 modified
dev/explanations/documentation-system/index.html	📝 modified
dev/explanations/extensions/index.html	📝 modified
dev/explanations/job_commands/index.html	📝 modified
dev/explanations/repo-structure/index.html	📝 modified
dev/explanations/run_demo/index.html	📝 modified
dev/explanations/tasks/index.html	➕ added
dev/explanations/testing/index.html	📝 modified
dev/explanations/web-architecture/index.html	📝 modified
dev/explanations/web-testing/index.html	📝 modified
dev/how-to/add-a-cli-command/index.html	📝 modified
dev/how-to/add-a-db/index.html	📝 modified
dev/how-to/add-a-route/index.html	📝 modified
dev/how-to/add-a-setting/index.html	📝 modified
dev/how-to/add-a-task/index.html	📝 modified
dev/how-to/add-a-test/index.html	📝 modified
dev/how-to/add-functionality/index.html	📝 modified
dev/how-to/client-customization/index.html	📝 modified
dev/how-to/client-extension/index.html	📝 modified
dev/how-to/client-generation/index.html	📝 modified
dev/how-to/contribute/index.html	📝 modified
dev/how-to/contribute-to-web/index.html	📝 modified
dev/how-to/create-web-application/index.html	📝 modified
dev/how-to/develop-legacy-dirac/index.html	📝 modified
dev/how-to/extend-diracx/index.html	📝 modified
dev/how-to/manage-web-extension/index.html	📝 modified
dev/how-to/setup-web-environment/index.html	📝 modified
dev/how-to/spawn-a-task/index.html	➕ added
dev/how-to/use-task-callbacks/index.html	➕ added
dev/how-to/use-the-demo/index.html	📝 modified
dev/how-to/write-docs/index.html	📝 modified
dev/reference/application-state/index.html	📝 modified
dev/reference/client-metapathfinder/index.html	📝 modified
dev/reference/coding-conventions/index.html	📝 modified
dev/reference/configuration/index.html	📝 modified
dev/reference/db-transaction-model/index.html	📝 modified
dev/reference/dependency-injection/index.html	📝 modified
dev/reference/entrypoints/index.html	📝 modified
dev/reference/env-variables/index.html	📝 modified
dev/reference/pixi-tasks/index.html	📝 modified
dev/reference/security-policies/index.html	📝 modified
dev/reference/security-properties/index.html	📝 modified
dev/reference/tasks/index.html	➕ added
dev/reference/test-recipes/index.html	📝 modified
dev/reference/web-coding-conventions/index.html	📝 modified
dev/reference/writing-tests/index.html	📝 modified
dev/tutorials/advanced-tutorial/index.html	📝 modified
dev/tutorials/getting-started/index.html	📝 modified
dev/tutorials/larger-developments/index.html	📝 modified
dev/tutorials/making-changes/index.html	➖ deleted
dev/tutorials/play-with-auth/index.html	➖ deleted
dev/tutorials/run-locally/index.html	📝 modified
dev/tutorials/web-extensions/index.html	📝 modified
dev/tutorials/web-getting-started/index.html	📝 modified
user/how-to/list-and-share-applications/index.html	📝 modified
user/how-to/login-out/index.html	📝 modified
user/how-to/monitor-jobs/index.html	📝 modified
user/reference/client-configuration/index.html	📝 modified
user/reference/known-installations/index.html	📝 modified
user/reference/programmatic-usage/index.html	📝 modified
user/tutorials/getting-started/index.html	📝 modified
admin/how-to/install/connect/index.html	📝 modified
admin/how-to/install/convert-cs/index.html	📝 modified
admin/how-to/install/embracing/index.html	📝 modified
admin/how-to/install/install-kubernetes/index.html	📝 modified
admin/how-to/install/installing/index.html	📝 modified
admin/how-to/install/minimal-requirements/index.html	📝 modified
admin/how-to/install/register-a-vo/index.html	📝 modified
admin/how-to/install/register-the-admin-vo/index.html	📝 modified
admin/how-to/tasks/configure/index.html	➕ added
admin/how-to/tasks/operate/index.html	➕ added
admin/how-to/tasks/run-task-manually/index.html	➕ added
dev/explanations/components/api/index.html	📝 modified
dev/explanations/components/cli/index.html	📝 modified
dev/explanations/components/client/index.html	📝 modified
dev/explanations/components/db/index.html	📝 modified
dev/explanations/components/routes/index.html	📝 modified
dev/explanations/tasks/broker-lifecycle/index.html	➕ added
dev/explanations/tasks/class-details/index.html	➕ added
dev/how-to/use-the-demo/swagger/index.html	📝 modified
dev/how-to/use-the-demo/web/index.html	📝 modified
dev/tutorials/advanced-tutorial/database/index.html	➕ added
dev/tutorials/advanced-tutorial/design/index.html	➕ added
dev/tutorials/advanced-tutorial/logic/index.html	➕ added
dev/tutorials/advanced-tutorial/router/index.html	➕ added
dev/tutorials/advanced-tutorial/running-locally/index.html	➕ added
dev/tutorials/advanced-tutorial/tasks/index.html	➕ added
dev/tutorials/advanced-tutorial/testing-database/index.html	➕ added
dev/tutorials/advanced-tutorial/testing-router/index.html	➕ added
dev/tutorials/advanced-tutorial/testing-tasks/index.html	➕ added
user/reference/programmatic-usage/command-line-interface/index.html	📝 modified
user/reference/programmatic-usage/https-interface/index.html	📝 modified
user/reference/programmatic-usage/python-interface/index.html	📝 modified

extensions/gubbins/pyproject.toml

fstagni · 2026-03-27T08:55:37Z

extensions/gubbins/pyproject.toml

 #    "gubbins-core/src/**/*.py",
    "gubbins-db/src/**/*.py",
    "gubbins-logic/src/**/*.py",
    "gubbins-routers/src/**/*.py",


Add

Suggested change

"gubbins-tasks/src/**/*.py",

fstagni · 2026-03-27T08:56:00Z

extensions/gubbins/pyproject.toml

 #    "$MYPY_CONFIG_FILE_DIR/gubbins-core/src",
    "$MYPY_CONFIG_FILE_DIR/gubbins-db/src",
    "$MYPY_CONFIG_FILE_DIR/gubbins-logic/src",
    "$MYPY_CONFIG_FILE_DIR/gubbins-routers/src",


Add

Suggested change

"$MYPY_CONFIG_FILE_DIR/gubbins-tasks/src",

fstagni · 2026-03-27T08:58:05Z

pixi.toml

 importlib-metadata = "<8.8.0"
+# Dependencies for run_local.sh
+redis-server = "*"
+seaweedfs = "*"


This is not part of this PR, right? Maybe refrain from adding it right now (it's also somewhere else in this PR)

I actually see it added in several places. Can you clarify?

Like it says in the comment: # Dependencies for run_local.sh

See https://diracx--842.org.readthedocs.build/en/842/dev/tutorials/advanced-tutorial/running-locally/ for more details.

aldbr · 2026-03-27T08:01:22Z

diracx-routers/src/diracx/routers/dependencies.py

-    _SandboxStoreSettings, Depends(_SandboxStoreSettings.create)
-]
+# Re-export everything from the canonical location
+from diracx.tasks.plumbing.depends import *  # noqa: F401, F403


Do we want diracx-routers to depend on diracx-tasks?
I would have naively imagined:
diracx-logic
|
-> diracx-tasks
|
-> diracx-routers

Note: there is a mermaid diagram in the documentation that would need to be adapted:

diracx/docs/dev/explanations/components/index.md

Lines 33 to 74 in 7cae3ca

```mermaid

---

config:

layout: elk

---

flowchart BT

subgraph frontend["Frontend"]

client["diracx-client (autorest)"]

api["diracx-api"]

cli["diracx-cli (typer)"]

end

subgraph backend["Backend"]

dbs["diracx-db (sqlalchemy/os)"]

logic["diracx-logic (Dirac)"]

routers["diracx-routers (FastAPI)"]

end

dbs -. uses .-> core["diracx-core (domain)"]

logic -. uses .-> core

routers -. uses .-> core

tasks["diracx-tasks (celery?)"] -. uses .-> core

client -. uses .-> core

api -. uses .-> core

cli -. uses .-> core

logic -- calls --> dbs

routers -- calls --> logic

tasks -- calls --> logic & api

client -- calls through OpenAPI --> routers

api -- calls --> client

cli -- calls --> api & client

client:::Sky

api:::Sky

cli:::Sky

dbs:::Pine

logic:::Pine

routers:::Pine

tasks:::Aqua

classDef Rose stroke-width:1px, stroke-dasharray:none, stroke:#FF5978, fill:#FFDFE5, color:#8E2236

classDef Sky stroke-width:1px, stroke-dasharray:none, stroke:#374D7C, fill:#E2EBFF, color:#374D7C

classDef Pine stroke-width:1px, stroke-dasharray:none, stroke:#254336, fill:#27654A, color:#FFFFFF

classDef Aqua stroke-width:1px, stroke-dasharray:none, stroke:#46EDC8, fill:#DEFFF8, color:#378E7A

```

https://diracx--842.org.readthedocs.build/en/842/adr/DX-ADR-001_tasks/#dependency-injection_1

extensions/gubbins/gubbins-tasks/src/gubbins/tasks/my_pilots.py

docs/dev/tutorials/larger-developments.md

Co-authored-by: Christophe Haen <christophe.haen@cern.ch>

…ents (DIRACGrid#810)

chrisburr force-pushed the feat/tasks branch from aac13ea to e3a8018 Compare March 20, 2026 15:58

chrisburr force-pushed the feat/tasks branch from e3a8018 to 9067077 Compare March 21, 2026 22:08

chrisburr mentioned this pull request Mar 24, 2026

docs: Add DiracX tasks architecture explanation #714

Closed

natthan-pigoux mentioned this pull request Mar 24, 2026

feat: create tasks to clean sandbox store chrisburr/diracx#1

Open

chrisburr force-pushed the feat/tasks branch from 9067077 to 8c5d7e5 Compare March 25, 2026 04:13

chrisburr changed the title ~~feat: Add diracx-tasks~~ feat: add diracx-tasks Mar 25, 2026

chrisburr force-pushed the feat/tasks branch 3 times, most recently from 81cfd3b to 36c857e Compare March 25, 2026 08:59

Loxeris reviewed Mar 25, 2026

View reviewed changes

extensions/gubbins/pyproject.toml Show resolved Hide resolved

chrisburr mentioned this pull request Mar 26, 2026

feat: replace container base images with pixi-managed environments #810

Merged

aldbr linked an issue Mar 26, 2026 that may be closed by this pull request

Tasks (replacement for agents/executors) #328

Open

3 tasks

chrisburr force-pushed the feat/tasks branch from c02b6e9 to 19b9ebc Compare March 26, 2026 22:16

chrisburr marked this pull request as ready for review March 26, 2026 22:17

chrisburr mentioned this pull request Mar 27, 2026

feat: add documentation for diracx container deploment #851

Draft

fstagni reviewed Mar 27, 2026

View reviewed changes

aldbr reviewed Mar 27, 2026

View reviewed changes

chrisburr and others added 3 commits March 27, 2026 13:04

feat: add diracx-tasks

7d65235

Co-authored-by: Christophe Haen <christophe.haen@cern.ch>

fixup! feat: replace container base images with pixi-managed environm…

a1e6f54

…ents (DIRACGrid#810)

chore: Update pixi.lock

65f2e76

chrisburr force-pushed the feat/tasks branch from 313b805 to 65f2e76 Compare March 27, 2026 12:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add diracx-tasks#842

feat: add diracx-tasks#842
chrisburr wants to merge 3 commits intoDIRACGrid:mainfrom
chrisburr:feat/tasks

chrisburr commented Mar 20, 2026

Uh oh!

chaen commented Mar 21, 2026

Uh oh!

read-the-docs-community bot commented Mar 21, 2026 •

edited

Loading

Uh oh!

Uh oh!

fstagni Mar 27, 2026

Uh oh!

fstagni Mar 27, 2026

Uh oh!

fstagni Mar 27, 2026

Uh oh!

fstagni Mar 27, 2026

Uh oh!

chrisburr Mar 27, 2026

Uh oh!

aldbr Mar 27, 2026

Uh oh!

aldbr Mar 27, 2026

Uh oh!

chrisburr Mar 27, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	```mermaid
	---
	config:
	layout: elk
	---
	flowchart BT
	subgraph frontend["Frontend"]
	client["diracx-client (autorest)"]
	api["diracx-api"]
	cli["diracx-cli (typer)"]
	end
	subgraph backend["Backend"]
	dbs["diracx-db (sqlalchemy/os)"]
	logic["diracx-logic (Dirac)"]
	routers["diracx-routers (FastAPI)"]
	end
	dbs -. uses .-> core["diracx-core (domain)"]
	logic -. uses .-> core
	routers -. uses .-> core
	tasks["diracx-tasks (celery?)"] -. uses .-> core
	client -. uses .-> core
	api -. uses .-> core
	cli -. uses .-> core
	logic -- calls --> dbs
	routers -- calls --> logic
	tasks -- calls --> logic & api
	client -- calls through OpenAPI --> routers
	api -- calls --> client
	cli -- calls --> api & client
	client:::Sky
	api:::Sky
	cli:::Sky
	dbs:::Pine
	logic:::Pine
	routers:::Pine
	tasks:::Aqua
	classDef Rose stroke-width:1px, stroke-dasharray:none, stroke:#FF5978, fill:#FFDFE5, color:#8E2236
	classDef Sky stroke-width:1px, stroke-dasharray:none, stroke:#374D7C, fill:#E2EBFF, color:#374D7C
	classDef Pine stroke-width:1px, stroke-dasharray:none, stroke:#254336, fill:#27654A, color:#FFFFFF
	classDef Aqua stroke-width:1px, stroke-dasharray:none, stroke:#46EDC8, fill:#DEFFF8, color:#378E7A

	```

Conversation

chrisburr commented Mar 20, 2026

Uh oh!

chaen commented Mar 21, 2026

Uh oh!

read-the-docs-community bot commented Mar 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

read-the-docs-community bot commented Mar 21, 2026 •

edited

Loading