diff --git a/.github/ISSUE_TEMPLATE/release-checklist.md b/.github/ISSUE_TEMPLATE/release-checklist.md index ca973c8c38..5e073cb59f 100644 --- a/.github/ISSUE_TEMPLATE/release-checklist.md +++ b/.github/ISSUE_TEMPLATE/release-checklist.md @@ -16,39 +16,17 @@ assignees: '' **Before release**: -- [ ] Make sure the release branch (e.g., `3.1.x`) is up to date with any backports. -- [ ] Make sure that all pull requests which will be included in the release have been properly documented as changelog files in the [`changes/` directory](https://github.com/zarr-developers/zarr-python/tree/main/changes). -- [ ] Run ``towncrier build --version x.y.z`` to create the changelog, and commit the result to the release branch. - [ ] Check [SPEC 0](https://scientific-python.org/specs/spec-0000/#support-window) to see if the minimum supported version of Python or NumPy needs bumping. -- [ ] Check to ensure that: - - [ ] Deprecated workarounds/codes/tests are removed. Run `grep "# TODO" **/*.py` to find all potential TODOs. - - [ ] All tests pass in the ["Tests" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/test.yml). - - [ ] All tests pass in the ["GPU Tests" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/gpu_test.yml). - - [ ] All tests pass in the ["Hypothesis" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/hypothesis.yaml). - - [ ] Check that downstream libraries work well (maintainers can make executive decisions about whether all checks are required for this release). - - [ ] numcodecs - - [ ] Xarray (@jhamman @dcherian @TomNicholas) - - Zarr's upstream compatibility is tested via the [Upstream Dev CI worklow](https://github.com/pydata/xarray/actions/workflows/upstream-dev-ci.yaml). - - Click on the most recent workflow and check that the `upstream-dev` job has run and passed. `upstream-dev` is not run on all all workflow runs. - - Check that the expected version of Zarr-Python was tested using the `Version Info` step of the `upstream-dev` job. - - If testing on a branch other than `main` is needed, open a PR modifying https://github.com/pydata/xarray/blob/90ee30943aedba66a37856b2332a41264e288c20/ci/install-upstream-wheels.sh#L56 and add the `run-upstream` label. - - [ ] Titiler.Xarray (@maxrjones) - - [Modify dependencies](https://github.com/developmentseed/titiler/blob/main/src/titiler/xarray/pyproject.toml) for titiler.xarray. - - Modify triggers for running [the test workflow](https://github.com/developmentseed/titiler/blob/61549f2de07b20cca8fb991cfcdc89b23e18ad05/.github/workflows/ci.yml#L5-L7). - - Push the branch to the repository and check for the actions for any failures. +- [ ] Verify that the latest CI workflows on `main` are passing: [Tests](https://github.com/zarr-developers/zarr-python/actions/workflows/test.yml), [GPU Tests](https://github.com/zarr-developers/zarr-python/actions/workflows/gpu_test.yml), [Hypothesis](https://github.com/zarr-developers/zarr-python/actions/workflows/hypothesis.yaml), [Docs](https://github.com/zarr-developers/zarr-python/actions/workflows/docs.yml), [Lint](https://github.com/zarr-developers/zarr-python/actions/workflows/lint.yml), [Wheels](https://github.com/zarr-developers/zarr-python/actions/workflows/releases.yml). +- [ ] Run the ["Prepare release" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/prepare_release.yml) with the target version. This will build the changelog and open a release PR with the `run-downstream` label. +- [ ] Verify that the [downstream tests](https://github.com/zarr-developers/zarr-python/actions/workflows/downstream.yml) (triggered automatically by the `run-downstream` label) pass on the release PR. +- [ ] Review the release PR and verify the changelog in `docs/release-notes.md` looks correct. +- [ ] Merge the release PR. **Release**: -- [ ] Go to https://github.com/zarr-developers/zarr-python/releases. - - [ ] Click "Draft a new release". - - [ ] Choose a version number prefixed with a `v` (e.g. `v0.0.0`). For pre-releases, include the appropriate suffix (e.g. `v0.0.0a1` or `v0.0.0rc2`). - - [ ] Set the target branch to the release branch (e.g., `3.1.x`) - - [ ] Set the description of the release to: `See release notes https://zarr.readthedocs.io/en/stable/release-notes.html#release-0-0-0`, replacing the correct version numbers. For pre-release versions, the URL should omit the pre-release suffix, e.g. "a1" or "rc1". - - [ ] Click on "Generate release notes" to auto-fill the description. - - [ ] Make a release by clicking the 'Publish Release' button, this will automatically create a tag too. -- [ ] Verify that release workflows succeeded. - - [ ] The latest version is correct on [PyPI](https://pypi.org/project/zarr/). - - [ ] The stable version is correct on [ReadTheDocs](https://zarr.readthedocs.io/en/stable/). +- [ ] [Draft a new GitHub Release](https://github.com/zarr-developers/zarr-python/releases/new) with tag `vX.Y.Z` targeting `main`. Use "Generate release notes" for the description. +- [ ] Verify the release is published on [PyPI](https://pypi.org/project/zarr/) and [ReadTheDocs](https://zarr.readthedocs.io/en/stable/). **After release**: @@ -57,3 +35,18 @@ assignees: '' --- - [ ] Party :tada: + +--- + +
+Releasing from a branch other than main + +In rare cases (e.g. patch releases for an older minor version), you may need to release from a dedicated release branch (e.g. `3.1.x`): + +- Create the release branch from the appropriate tag if it doesn't already exist. +- Cherry-pick or backport the necessary commits onto the branch. +- Run `towncrier build --version x.y.z` and commit the result to the release branch instead of `main`. +- When drafting the GitHub Release, set the target to the release branch instead of `main`. +- After the release, ensure any relevant changelog updates are also reflected on `main`. + +
diff --git a/.github/workflows/downstream.yml b/.github/workflows/downstream.yml new file mode 100644 index 0000000000..c93ab94c49 --- /dev/null +++ b/.github/workflows/downstream.yml @@ -0,0 +1,105 @@ +name: Downstream + +on: + workflow_dispatch: + pull_request: + types: [labeled] + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + xarray: + name: Xarray zarr backend tests + if: github.event_name == 'workflow_dispatch' || github.event.label.name == 'run-downstream' + runs-on: ubuntu-latest + steps: + - name: Check out zarr-python + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + fetch-depth: 0 + persist-credentials: false + + - name: Check out xarray + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + repository: pydata/xarray + path: xarray + persist-credentials: false + + - name: Set up pixi + uses: prefix-dev/setup-pixi@19eac09b398e3d0c747adc7921926a6d802df4da # v0.8.8 + with: + manifest-path: xarray/pixi.toml + + - name: Install zarr-python from branch + working-directory: xarray + run: pixi run -e test-py313 -- pip install --no-deps .. + + - name: Show versions + working-directory: xarray + run: | + pixi run -e test-py313 -- python -c " + import zarr; print(f'zarr {zarr.__version__}') + import xarray; print(f'xarray {xarray.__version__}') + " + + - name: Run xarray zarr backend tests + working-directory: xarray + run: | + pixi run -e test-py313 -- python -m pytest -x --no-header -q \ + xarray/tests/test_backends.py -k zarr \ + xarray/tests/test_backends_api.py -k zarr \ + xarray/tests/test_backends_datatree.py -k zarr + + numcodecs: + name: numcodecs zarr3 codec tests + if: github.event_name == 'workflow_dispatch' || github.event.label.name == 'run-downstream' + runs-on: ubuntu-latest + steps: + - name: Check out zarr-python + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + fetch-depth: 0 + persist-credentials: false + + - name: Check out numcodecs + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + repository: zarr-developers/numcodecs + fetch-depth: 0 + path: numcodecs + persist-credentials: false + + - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 + with: + python-version: '3.13' + + - name: Install uv + uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7 + + - name: Install numcodecs with test-zarr-main group + working-directory: numcodecs + run: | + uv sync --group dev --group test-zarr-main + uv pip install --no-build-isolation -e . + + - name: Override zarr-python with branch version + working-directory: numcodecs + run: uv pip install --no-deps .. + + - name: Show versions + working-directory: numcodecs + run: | + uv run python -c " + import zarr; print(f'zarr {zarr.__version__}') + import numcodecs; print(f'numcodecs {numcodecs.__version__}') + " + + - name: Run numcodecs zarr3 tests + working-directory: numcodecs + run: uv run python -m pytest -x --no-header -q tests/test_zarr3.py diff --git a/.github/workflows/prepare_release.yml b/.github/workflows/prepare_release.yml new file mode 100644 index 0000000000..4bccb40092 --- /dev/null +++ b/.github/workflows/prepare_release.yml @@ -0,0 +1,77 @@ +name: Prepare release notes + +on: + workflow_dispatch: + inputs: + version: + description: 'Release version notes (e.g. 3.2.0)' + required: true + type: string + target_branch: + description: 'Branch to target' + required: false + default: 'main' + type: string + +permissions: + contents: write + pull-requests: write + +jobs: + prepare: + name: Build changelog and open PR + runs-on: ubuntu-latest + steps: + - name: Validate inputs + run: | + if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+([-\.][a-zA-Z0-9]+)*$ ]]; then + echo "::error::Invalid version format: '$VERSION'" + exit 1 + fi + if [[ ! "$TARGET_BRANCH" =~ ^[a-zA-Z0-9._/-]+$ ]]; then + echo "::error::Invalid branch name: '$TARGET_BRANCH'" + exit 1 + fi + env: + VERSION: ${{ inputs.version }} + TARGET_BRANCH: ${{ inputs.target_branch }} + + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + ref: ${{ inputs.target_branch }} + fetch-depth: 0 + persist-credentials: false + + - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 + with: + python-version: '3.12' + + - name: Install towncrier + run: pip install towncrier + + - name: Build changelog + run: towncrier build --version "$VERSION" --yes + env: + VERSION: ${{ inputs.version }} + + - name: Create pull request + uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8 + with: + branch: release/v${{ inputs.version }} + base: ${{ inputs.target_branch }} + title: "Release v${{ inputs.version }}" + body: | + Automated release preparation for v${{ inputs.version }}. + + This PR was generated by the "Prepare release" workflow. It includes: + - Rendered changelog via `towncrier build --version ${{ inputs.version }}` + - Removal of consumed changelog fragments from `changes/` + + ## Checklist + + - [ ] Review the rendered changelog in `docs/release-notes.md` + - [ ] Downstream tests pass (see [downstream workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/downstream.yml)) + - [ ] Merge this PR, then [draft a GitHub Release](https://github.com/zarr-developers/zarr-python/releases/new) targeting `${{ inputs.target_branch }}` with tag `v${{ inputs.version }}` + commit-message: "chore: build changelog for v${{ inputs.version }}" + labels: run-downstream + delete-branch: true diff --git a/docs/contributing.md b/docs/contributing.md index e62ce54c35..d44b58992c 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -316,6 +316,16 @@ If an existing Zarr format version changes, or a new version of the Zarr format Open an issue on GitHub announcing the release using the release checklist template: [https://github.com/zarr-developers/zarr-python/issues/new?template=release-checklist.md](https://github.com/zarr-developers/zarr-python/issues/new?template=release-checklist.md). The release checklist includes all steps necessary for the release. +### Preparing a release + +Releases are prepared using the ["Prepare release notes"](https://github.com/zarr-developers/zarr-python/actions/workflows/prepare_release.yml) workflow. To run it: + +1. Go to the [workflow page](https://github.com/zarr-developers/zarr-python/actions/workflows/prepare_release.yml) and click "Run workflow". +2. Enter the release version (e.g. `3.2.0`) and the target branch (defaults to `main`). +3. The workflow will run `towncrier build` to render the changelog, remove consumed fragments from `changes/`, and open a pull request on the `release/v` branch. +4. The release PR is automatically labeled `run-downstream`, which triggers the [downstream test workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/downstream.yml) to run Xarray and numcodecs integration tests against the release branch. +5. Review the rendered changelog in `docs/release-notes.md` and verify downstream tests pass before merging. + ## Benchmarks Zarr uses [pytest-benchmark](https://pytest-benchmark.readthedocs.io/en/latest/) for running