Diffusion ci tests

[CharlelieLrt]

PhysicsNeMo Pull Request

Description

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.
• The CHANGELOG.md is up to date with these changes.
• An issue is linked to this pull request.
• If I am implementing a new model or modifying any existing model, I have followed the Models Implementation Coding Standards.

Dependencies

Review Process

All PRs are reviewed by the PhysicsNeMo team before merging.

Depending on which files are changed, GitHub may automatically assign a maintainer for review.

We are also testing AI-based code review tools (e.g., Greptile), which may add automated comments with a confidence score.
This score reflects the AI’s assessment of merge readiness and is not a qualitative judgment of your work, nor is
it an indication that the PR will be accepted / rejected.

AI-generated feedback should be reviewed critically for usefulness.
You are not required to respond to every AI comment, but they are intended to help both authors and reviewers.
Please react to Greptile comments with 👍 or 👎 to provide feedback on their accuracy.

[CharlelieLrt]

/blossom-ci

[coreyjadams]

Hey @greptile this pr introduces nearly 400 binary files for numerical checkpoints and consistency, which might complicate git cloning. Got any suggestions for handling that?

[greptile-apps]

Here are a few approaches to handle those binary checkpoint files:

1. Git LFS (most common)

• Move binary files to Git LFS - keeps them out of clone history
• Add to .gitattributes: *.npy filter=lfs diff=lfs merge=lfs -text (or whatever format you're using)
• Clones stay fast, files downloaded only when needed
• CI can use GIT_LFS_SKIP_SMUDGE=1 to skip download, then selectively fetch

2. External storage with download script

• Store checkpoints in S3/GCS/Azure blob storage
• Add a scripts/download_test_fixtures.py that fetches them before tests
• CI downloads on demand, developers only download if running those specific tests
• Keep a manifest file (JSON/YAML) tracking what's available

3. Hybrid: essential vs full test suites

• Keep a minimal set (~10-20 files) for basic CI checks in repo
• Store the full 400-file suite externally for comprehensive testing
• Run full suite only on release branches or manually triggered workflows

4. Programmatic generation

• If checkpoints are deterministic, generate them in CI from smaller seed files
• Only store the configuration/seeds (much smaller)
• Trade storage for compute time

For PhysicsNeMo specifically, I'd suggest Git LFS + lazy loading in CI (option 1) since it:

• Maintains reproducibility (files versioned with code)
• Keeps clones fast
• Integrates cleanly with existing workflows

What kind of files are these - numpy arrays, model weights, HDF5?

[CharlelieLrt]

/blossom-ci

[CharlelieLrt]

/blossom-ci