⚡ Improve `Engine` Performance and Implementation by shaneahmed · Pull Request #578 · TissueImageAnalytics/tiatoolbox

shaneahmed · 2023-03-31T09:55:58Z

Summary

This PR fundamentally refactors TIAToolbox's model/engine infrastructure into a unified, optimized architecture with a consistent run() method interface across all engines. It introduces efficient zarr/dask-based processing for large-scale WSI analysis, multiple flexible output formats, and three state-of-the-art pre-trained models: GrandQC, Tissue Mask, and KongNet. Additionally, improved CLIs, comprehensive test coverage, and updated documentation ensure a smooth developer experience.

What's New

1. Unified Engine Architecture

New EngineABC base class providing a standardized interface for all engines
All engines now inherit from EngineABC and implement the consistent run() method
Engines moved to dedicated tiatoolbox.models.engines module
Standardized parameter passing and error handling across all engines

2. Optimized Data Processing

WSI processing now uses the same WSI-based dataloader as patch processing for consistency
Intermediate outputs saved as zarr format to resolve memory issues with large WSIs
Dask integration for lazy evaluation and efficient memory management
Tile-based processing with configurable tile shapes and memory thresholds
New postprocessing utilities for peak detection, NMS, and overlap-aware computations

3. Flexible Output Types

Standardized output options: dict, zarr, annotationstore, qupath JSON
Model outputs now consistently return dictionaries for better consistency
Support for AnnotationStore output enables visualization using TIAViz
Each output type validated and tested comprehensively

4. Enhanced Model Architectures

Improved Core Architectures:

HoverNet & HoverNetPlus - Updated for Dask-aware postprocessing with offset-aware instance handling
SCCNN & MAPDE - Switched to overlap-aware peak detection for improved accuracy
MicroNet - Enhanced with Dask support and standardized output dictionaries
UNet - Aligned inference output handling and added postprocessing standardization
Vanilla architectures - Refactored backbone registries and shared postprocessing utilities

5. New Engine Implementations

PatchPredictor - Redesigned with EngineABC base class
DeepFeatureExtractor - Updated API for foundation models (UNI, Prov-GigaPath, H-optimus-0) with zarr output
MultiTaskSegmentor - Enhanced performance for multi-task learning scenarios
NucleusDetector - Complete nucleus detection and localization engine across various tissue types
NucleusInstanceSegmentor - Instance-level nucleus segmentation with updated interface
SemanticSegmentor - Semantic segmentation engine with flexible output options

6. New Pre-trained Models

GrandQC

Quality control model for histopathology image quality assessment
Integrated with tissue quality prediction pipeline
Enables automated quality filtering in analysis workflows

Tissue Mask (EfficientUNet-based)

Automated tissue segmentation and background removal
Available across multiple resolutions: 0.25 mpp baseline, 0.5 mpp, and 1.0 mpp
Lightweight and efficient for rapid WSI preprocessing

KongNet

Multi-task nucleus detection and classification model
State-of-the-art architecture achieving top rankings in Grand Challenges (MONKEY, MIDOG, PUMA)
Pre-trained models for multiple datasets:
- KongNet_CoNIC_1 - Nucleus segmentation and classification (0.5 mpp)
- KongNet_PanNuke_1 - Multi-class nucleus detection on PanNuke dataset (0.25 mpp)
- KongNet_MONKEY_1 - Immune cell detection in primate tissue (0.25 mpp)
- KongNet_PUMA_T1_3 & KongNet_PUMA_T2_3 - Multi-headed nucleus detection variants (0.25 mpp)
- KongNet_Det_MIDOG_1 - Mitosis detection with object detection variant (0.5 mpp)

7. Enhanced Configuration

Unified ModelIOConfigABC for input/output specifications
Support for IOPatchPredictorConfig and IOSegmentorConfig
return_probabilities option added to engine parameters
post_process_cache_mode for WSI-level algorithm execution
Updated documentation with model-specific I/O configurations

8. Improved CLI Infrastructure

New CLI commands: semantic-segmentor, multitask-segmentor, nucleus-detector, deep-feature-extractor
Updated CLI commands: patch-predictor, nucleus-instance-segment migrated to new run() API
Unified CLI option handling across all engines
Consistent logging and progress tracking
Support for all output types via command-line arguments

Key Changes

API Changes:

Removed merge_predictions option from engines (replaced with more efficient architecture)
Removed save_wsi_output parameter (handled automatically after post-processing)
Fixed docstrings in EngineABCRunParams
Added infer_wsi() method for WSI inference support
NucleusDetector provides efficient nucleus detection with centroid mapping and post-processing
Model attribute access via _get_model_attr() for compatibility checks

Performance Improvements:

compile_model now moved to EngineABC initialization phase
Fixed bug with _calculate_scale_factor for proper resolution scaling
Fixed bug in class_dict definition for multi-task segmentation
patch_predictions_as_annotations now loops on patch_coords instead of class_probs for efficiency
KongNet models optimized for both single-image and batch WSI processing
Overlap-aware peak detection for improved accuracy in dense regions
Non-Maximum Suppression (NMS) utilities for efficient post-processing

Utility Updates:

Made _get_zarr_array() a public function get_zarr_array() in misc module
Added postprocessing utilities: detect_peaks_dask, apply_nms, etc. in architecture/utils.py
Support for interactive visualization with TIAViz via AnnotationStore output
Enhanced logging and progress tracking with tqdm
Dask-aware utilities for handling overlapping regions and tile merging

Test Coverage:

Comprehensive test suite for new engine architecture (tests/engines/)
- Tests for all output types (dict, zarr, annotationstore, qupath)
- Tests for both patch and WSI processing modes
- Tests for multi-GPU feature extraction
- Tests for post-processing utilities
CLI tests for all engine commands
Updated architecture tests for new postprocessing signatures
Tests for NucleusDetector with various model configurations
Tests for new KongNet models across different datasets
Parameter validation and error handling tests

Documentation:

Updated example notebooks to use new API
Fixed docstring references in model configurations
Added comprehensive documentation for new models (GrandQC, Tissue Mask, KongNet)
Updated README references to point to official documentation
Added notebook examples for nucleus detection (12. Nuclei Detection and Classification)
Updated pretrained model documentation with I/O configuration examples

Data Source Updates:

Updated test data URLs from legacy servers to HuggingFace Hub
Added support for downloading new model weights and sample data
Improved sample datasets for KongNet and nucleus detection
New thumbnail samples for testing foundation models

Dependencies:

Added dask>=2026.1.2 for distributed computation and lazy evaluation
Added ipywidgets>=8.1.7 for notebook interactions and progress visualization
Updated data sources to use HuggingFace Hub instead of legacy servers

Migration Notes

Users upgrading to this version should:

Update engine instantiation to use run() method instead of predict()
Specify output types explicitly (defaults to dict)
Use patch_mode=False for WSI processing
Provide save_dir when using zarr or annotationstore output types
For nucleus detection tasks, use the new NucleusDetector engine with pre-trained models
Update imports from tiatoolbox.models.engine to tiatoolbox.models.engines
Consult updated documentation for model-specific I/O configurations

Example Usage

# Using NucleusDetector with pre-trained models
from tiatoolbox.models.engines import NucleusDetector

detector = NucleusDetector(model="sccnn-conic")
output = detector.run(
    images=[wsi_path],
    patch_mode=False,
    device="cuda",
    save_dir="./output",
    output_type="annotationstore"
)

# Using DeepFeatureExtractor with foundation models
from tiatoolbox.models.engines import DeepFeatureExtractor

extractor = DeepFeatureExtractor(model="UNI", batch_size=32)
features = extractor.run(
    images=[wsi_path],
    patch_mode=False,
    device="cuda",
    output_type="zarr",
    save_dir="./features"
)

# Using MultiTaskSegmentor with advanced output options
from tiatoolbox.models.engines import MultiTaskSegmentor

segmentor = MultiTaskSegmentor(model="hovernetplus-oed")
results = segmentor.run(
    images=[wsi_path],
    patch_mode=False,
    device="cuda",
    output_type="qupath",
    save_dir="./segmentation",
    return_predictions=(True, True),
    return_probabilities=True
)

# Using GrandQC for quality assessment
from tiatoolbox.models.engines import SemanticSegmentor

qc_engine = SemanticSegmentor(model="grandqc")
quality_scores = qc_engine.run(
    images=[image_paths],
    patch_mode=True,
    output_type="dict"
)

- Use `pyproject.toml` for `bdist_wheel` configuration

…-abc

- Improve `Engines` performance and implementation

codecov · 2023-03-31T10:31:07Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.53%. Comparing base (25430d5) to head (b6815f3).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #578      +/-   ##
===========================================
+ Coverage    99.41%   99.53%   +0.12%     
===========================================
  Files           72       83      +11     
  Lines         9540    11354    +1814     
  Branches      1267     1493     +226     
===========================================
+ Hits          9484    11301    +1817     
+ Misses          29       28       -1     
+ Partials        27       25       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Refactor engines_abc.py

for more information, see https://pre-commit.ci

## Summary This PR integrates KongNet, a state‑of‑the‑art multi‑headed deep learning architecture for nucleus detection and classification. It adds complete documentation, examples, and notebook support covering both patch‑based and whole‑slide image (WSI) processing. Multiple pretrained models are now available for major pathology datasets, each demonstrated through updated example notebooks. The PR also introduces a new visualization image and practical usage patterns for the NucleusDetector class, enabling streamlined workflows for researchers using TIAToolbox. ## What’s New ### 🧠 KongNet Model Integration Added pretrained KongNet models: - KongNet_CoNIC_1, KongNet_PanNuke_1, KongNet_MONKEY_1, - KongNet_PUMA_T1_3, KongNet_PUMA_T2_3, KongNet_Det_MIDOG_1. - These models capture multi-headed detection signals and achieve strong performance on CoNIC, PanNuke, MONKEY, PUMA, and MIDOG 2025 datasets. ### 📘 New Example Notebook: Nucleus Detection A comprehensive example notebook demonstrates: - Patch‑level and WSI‑level detection - Mitosis and immune‑cell detection - Use of NucleusDetector with KongNet pretrained models - Interactive visualization via TIAViz - [x] Add KongNet Model - [x] Add tests - [x] Add example notebook (Nucleus Detector) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Co-authored-by: Jiaqi Lv <jiaqilv@Jiaqis-MacBook-Pro.local>

## Summary This PR introduces a full **EfficientNet‑UNet Tissue Mask Model** implementation. It adds the EfficientNet‑B0 encoder, UNet‑style decoder, segmentation head, and utilities for preprocessing, postprocessing, and batch inference. The model enables high‑quality binary tissue segmentation with ImageNet normalization and morphology‑based cleanup. ## What's New - New architecture components: - EfficientNet‑B0 encoder (MBConv blocks, SAME‑padding convs) - UNet decoder with skip connections - Segmentation head - `EfficientUNetTissueMaskModel` with `preproc`, `postproc`, and `infer_batch` - Integration with `SemanticSegmentor` for direct WSI‑scale tissue masking. ## Examples ### 1. Using with `SemanticSegmentor` ```python from pathlib import Path from tiatoolbox.models.engine.semantic_segmentor import SemanticSegmentor segmentor = SemanticSegmentor(model="efficientunet-tissue_mask") outputs = segmentor.run( images=["/path/to/wsi.svs"], patch_mode=False, output_type="annotationstore", save_dir=Path("./tissue_mask_outputs"), ) ``` ### 2. Direct model usage ```python import numpy as np, torch from tiatoolbox.models.architecture.efficientunet_tissue_mask_model import EfficientUNetTissueMaskModel model = EfficientUNetTissueMaskModel(num_classes=1) img = np.ones((256, 256, 3), dtype=np.uint8) * 128 processed = model.preproc(img) batch = torch.from_numpy(processed[None]).float() probs = model.infer_batch(model, batch, device="cpu") mask = model.postproc(probs[0]) ``` --------- Co-authored-by: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com>

…1000) ## Summary This PR modernises the annotation workflow by moving fully to SQLite (`.db`) annotation stores and updates the HoverNet box‑segmentation path to use the current API. Test coverage is expanded accordingly. ## Key Changes - Add tests for annotation saving and HoverNet execution on a selected box region. - Standardise annotation handling by converting and loading `.db` stores instead of legacy `.dat` files. - Update HoverNet integration to use the newer `run()` interface and revised model parameters. - Simplify TileServer ingestion logic by removing the old `add_from_dat` path and loading annotations directly from SQLite stores. - Reverts `Bokeh` Tests for Compatibility with the New Engine Design --------- Co-authored-by: measty <20169086+measty@users.noreply.github.com>

…1002) ## Summary This PR updates the multi‑task segmentation example notebook to align with the latest TIAToolbox API and data sources. It cleans up metadata, modernizes API usage, and switches all sample downloads to stable HuggingFace hosting. ## Key Changes - Removed deprecated metadata (`remove-cell` tags, old IDs). - Updated documentation links and installation notes. - Updated dependencies: removed `joblib`, added `zarr`, adjusted apt packages. - Switched sample image/WSI downloads to HuggingFace. - Migrated to new TIAToolbox API: - `predict()` → `run()` - `pretrained_model` → `model` - `num_loader_workers` → `num_workers` - `mode` → `patch_mode` - Added new args (`wsireader_kwargs`, `return_predictions`, `overwrite`) - Output now saved as Zarr with keys: `layer_segmentation`, `nuclei_segmentation`.

- Update Jupyter Notebook `07-advanced-modeling.ipynb`

- Update `idars` notebook and clean outputs in multitask segmentor and nuclei instance segmentor --------- Co-authored-by: Jiaqi Lv <lvjiaqi9@gmail.com>

- Update slide-graph notebooks

- Refactor `models.engine` to `models.engines` for Consistency

## Summary This PR updates the WSI segmentation workflow so that whole‑slide images are processed **only within the tissue‑mask bounds**, and the resulting predictions are **padded back to full-slide dimensions** once inference completes. It also adds support for supplying a global **offset** to all model post‑processing steps, ensuring instance and semantic outputs (centroids, contours, boxes) remain aligned with the original slide coordinates. The dataset/engine API is made more consistent (`mask_path` → `input_mask`), and tests are expanded to cover masked, non-square, and tile-based scenarios. ## Key Changes ### Mask-bounded processing and output padding - Engines compute `mask_bounds`, filter output locations to those inside the mask, and track required padding. - Final probabilities and predictions are padded using Dask to restore the original full-slide output shape. - Supports both semantic and multi-task segmentation paths. ### Offset-aware post-processing - `HoVerNet`, `HoVerNetPlus`, and `MicroNet` now accept an `offset` argument and apply it to all geometric outputs. - Engines automatically provide the appropriate offset based on mask origin or tile layout. ### API and docstring updates - `mask_path` is renamed to `input_mask` and now accepts either a file path or an `np.ndarray` mask. - Docstrings clarify default behaviors for `auto_get_mask`, probabilities, memory thresholds, and worker counts. - Type hints and parameter descriptions improved for consistency across dataset and engine classes. ### Tile merging and instance ID handling - Horizontal merge functions support a fixed `canvas_width` to avoid allocating overly large row canvases. - Tile-based instance merging now preserves unique instance IDs across tiles and handles overlaps more safely. ### Test updates - Tests updated to use `input_mask`, verify masked execution paths, validate shapes and probability behavior, and cover tile mode and offset-aware post-processing.

Copilot

Pull request overview

This PR refactors TIAToolbox’s model/engine stack into a unified engines API with a consistent run() interface, adds new nucleus detection/segmentation capabilities, and updates CLIs/tests/docs to match the new architecture and improve large-WSI performance.

Changes:

Introduces unified engine/IO-config interfaces + new CLI commands for segmentation, detection, and feature extraction.
Adds optimized post-processing utilities (Dask overlap peak detection + NMS) and updates model architectures to standardized outputs.
Updates test suite, sample data sources, and example/docs references for the new API.

Reviewed changes

Copilot reviewed 66 out of 102 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
tiatoolbox/models/dataset/init.py	Re-exports dataset APIs with explicit `__all__`.
tiatoolbox/models/architecture/vanilla.py	Refactors backbone registries + shared postproc utility usage.
tiatoolbox/models/architecture/utils.py	Adds postproc utilities (peak detection, NMS) and UNet++ helper modules.
tiatoolbox/models/architecture/unet.py	Aligns inference output to ndarray and adds argmax postproc.
tiatoolbox/models/architecture/sccnn.py	Switches to overlap-aware peak detection and standardizes infer/postproc signatures.
tiatoolbox/models/architecture/micronet.py	Adds Dask-aware postproc + standardizes outputs to task dicts.
tiatoolbox/models/architecture/mapde.py	Switches to overlap-aware peak detection and standardizes infer/postproc signatures.
tiatoolbox/models/architecture/hovernetplus.py	Adds Dask-aware postproc output dictionaries + offset handling.
tiatoolbox/models/architecture/hovernet.py	Adds Dask-support helpers + offset-aware instance info, standardized return format.
tiatoolbox/models/architecture/init.py	Updates pretrained model loading and adds backbone shortcut support.
tiatoolbox/models/init.py	Switches public exports from `engine` to `engines` and adds new engines/models.
tiatoolbox/data/remote_samples.yaml	Adds new HF-hosted thumbnail sample.
tiatoolbox/cli/tissue_mask.py	Tightens `--mode` choices.
tiatoolbox/cli/semantic_segmentor.py	New semantic segmentation CLI using new engine API.
tiatoolbox/cli/semantic_segment.py	Removes legacy semantic segmentation CLI.
tiatoolbox/cli/patch_predictor.py	Migrates CLI to new `run()` engine API + unified options.
tiatoolbox/cli/nucleus_instance_segment.py	Migrates CLI to new `run()` API + unified options.
tiatoolbox/cli/nucleus_detector.py	Adds nucleus detector CLI.
tiatoolbox/cli/multitask_segmentor.py	Adds multitask segmentor CLI.
tiatoolbox/cli/deep_feature_extractor.py	Adds deep feature extractor CLI.
tiatoolbox/cli/init.py	Registers new CLI commands and renames semantic seg command.
tests/test_wsireader.py	Updates remote sample URL to Hugging Face.
tests/test_tileserver.py	Updates tile server tests to use SQLiteStore via `store_from_dat`.
tests/test_cli.py	Adds tests for new CLI parsing utilities.
tests/test_app_bokeh.py	Adjusts save/load annotation layer test ordering and cleanup.
tests/models/test_multi_task_segmentor.py	Removes legacy multitask segmentor tests (replaced elsewhere).
tests/models/test_models_abc.py	Adds `get_pretrained_model` error-path tests.
tests/models/test_hovernetplus.py	Updates for new postproc return structure + offset parameter.
tests/models/test_hovernet.py	Updates for new postproc return structure + offset parameter.
tests/models/test_feature_extractor.py	Removes legacy feature extractor tests (replaced in engines tests).
tests/models/test_dataset.py	Expands dataset tests for new dataset/WSI patch dataset behaviors.
tests/models/test_arch_vanilla.py	Updates postproc test data types.
tests/models/test_arch_utils.py	Adds tests for new peak detection, NMS, and attention module utilities.
tests/models/test_arch_timm_efficientnet.py	Adds unit tests for timm EfficientNet encoder helpers.
tests/models/test_arch_sccnn.py	Updates tests for new SCCNN peak-map postproc output.
tests/models/test_arch_micronet.py	Migrates test to new NucleusInstanceSegmentor + zarr output assertions.
tests/models/test_arch_mapde.py	Updates tests for new peak-map postproc output + override params.
tests/models/test_arch_kongnet.py	Adds extensive unit tests for new KongNet model and detector integration.
tests/models/test_arch_grandqc.py	Adds GrandQC model tests + SemanticSegmentor integration checks.
tests/models/test_arch_efficientnetunet_tissue_mask_model.py	Adds EfficientUNet tissue mask model tests + integration checks.
tests/engines/test_nucleus_instance_segmentor.py	Adds engine + CLI tests for nucleus instance segmentation.
tests/engines/test_nucleus_detection_engine.py	Adds engine + CLI tests for nucleus detection and output types.
tests/engines/test_ioconfig.py	Adds validation and conversion tests for IOConfig models.
tests/engines/test_feature_extractor.py	Adds engine + CLI tests for deep feature extraction in patch/WSI modes.
tests/engines/init.py	Adds `tests.engines` package marker.
tests/conftest.py	Adds new WSI sample + rm_dir fixture.
requirements/requirements.txt	Adds new dependencies for Dask + ipywidgets.
examples/full-pipelines/slide-graph.ipynb	Updates install/docs links + adds incompatibility notice for v2+.
examples/README.md	Adds new nucleus detection example entry.
examples/11-import-foundation-models.ipynb	Migrates notebook to new engines API + HF sample download + zarr outputs.
examples/04-patch-extraction.ipynb	Updates sample URLs to HF dataset and adjusts cells/metadata.
docs/pretrained.rst	Updates pretrained model IOConfig examples and extends KongNet documentation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Jiaqi-Lv · 2026-03-04T15:11:09Z

@copilot Can you generate a summary of this PR?

Copilot · 2026-03-04T15:11:28Z

@Jiaqi-Lv I've opened a new pull request, #1011, to work on those changes. Once the pull request is ready, I'll request review from you.

* Revert "♻️ Refactor `models.engine` to `models.engines` for Consistency (#1009)" This reverts commit 4ac2245. * ⏪ Revert `engines` to `engine` API change

@aacic

## TIAToolbox v2.0.0 (2026-03-11) ### ✨ Major Updates and Feature Improvements #### ⚙️ Engine Redesign (PR #578) TIAToolbox 2.0.0 introduces a completely re-engineered inference engine designed for significant performance, scalability, and memory-efficiency improvements. #### Key Enhancements - A modern processing stack built on **Dask** (parallel/distributed execution) and **Zarr** (chunked, out-of-core storage) - **Standardised output formats** across all engines: - Python `dict` - **Zarr** - **AnnotationStore** (SQLite-backed) - **QuPath JSON** - Cleaner runtime behavior with reduced warning noise and a unified progress bar - More predictable memory usage through chunked streaming - Broader test coverage across engine components ### 🗺️ Improved QuPath Support Enhancements include: - Better handling of **GeoJSON** - Support for **multipoint geometries** (#841) - Improved semantic output helpers: - `dict_to_store_semantic_segmentor` (#926) - OME-TIFF probability overlays (#929) ### 🔬 New Nucleus Detection Engine A dedicated nucleus detection pipeline has been added, built on the redesigned engine for improved accuracy and efficient large-scale processing. #### 🧠 KongNet Model Family TIAToolbox 2.0.0 introduces **KongNet**, a high-performance architecture that achieved top results across multiple international challenges: - 🥇 **1st place: MONKEY Challenge (overall detection)** - 🥇 **1st place: MIDOG (mitosis detection)** - ⭐ Top-tier performance on **PUMA** Multiple pretrained variants are available (CoNIC, PanNuke, MONKEY, PUMA, MIDOG), each with standardised IO configurations. ### 🧬 Expanded Foundation Model Support Additional foundation models are now supported (#906), broadening the range of high-capacity architectures available for feature extraction and downstream tasks. ### 🖼️ SAM Segmentation in TIAViz TIAViz now integrates Meta’s Segment Anything Model (SAM), enabling: - Interactive segmentation - Rapid region extraction - Exploratory annotation workflows Simplified SAM usage (#968) streamlines its integration into analysis pipelines. ### 🧩 Enhanced WSIReader & Metadata Handling Major improvements include: - More robust cross-vendor **metadata extraction** (#1001) - **Multichannel image support** (PR #825) for immunofluorescence and non-RGB modalities - Simplified Windows installation using `openslide-bin` (no manual DLL steps) - macOS Tileserver fix (#976) - Improved DICOM reading (#934) ### ☁️ New Cloud-Native Reader: FsspecJSONWSIReader (PR #897) A new reader supporting **fsspec-compatible filesystems**, enabling seamless access to WSIs stored on: - S3 - GCS - Azure - HPC clusters - Any fsspec-supported backend This enables cloud-native and distributed data workflows. Contributed by @aacic ### 🤗 Pretrained Models Migrated to Hugging Face All pretrained models and sample assets have been migrated (#945, #983), improving: - Download reliability - Versioning and reproducibility - Caching and CI integration - Licensing clarity per model family ### 🛡️ Security, Compatibility & Tooling #### 🔐 Security & Dependency Updates - Dependency upgrades - Internal security improvements - Explicit workflow permissions added (#1021, #1023) #### 🐍 Python Version Support - **Dropped:** Python **3.9** - **Added:** Python **3.13** - **Supported:** Python 3.10–3.13 - Updated CUDA wheel source to **cu126** #### 🛠️ Developer Tooling & CI/CD - Expanded **mypy** type-checking coverage (#912, #931, #935, #951) - Updated pre-commit hooks and general formatting - CI uses **CPU-only PyTorch** for faster, more reliable builds (#974, #979) - Updated pip install workflow (#1013) - Added new **Python 3.13 Docker images** (#1014, #1019) ### 🧹 Bug Fixes & Stability Improvements - Fixed multi-GPU behaviour with `torch.compile` (#923) - Fixed DICOM reading issue (#934) - Fixed annotation contour handling with holes (#956) - Fixed consecutive annotation load bug (#927) - Fixed SCCNN model issues (#970) - Fixed MapDe `dist_filter` shape issue (#914) - Improved notebook reliability on Colab (#1026–#1030) - macOS TileServer issues resolved (#976) ### 🧭 Migration Guide for Users #### 🔄 Updating from 1.x to 2.0.0 #### Update calls: replace `.predict()` with `.run()` ```python # Old results = segmentor.predict(imgs=[...], ioconfig=config) # New results = segmentor.run(images=[...], ioconfig=config) ``` #### Use `patch_mode`: replace `mode="patch"` with `patch_mode=True` and `mode="tile"` or "wsi" with `patch_mode=False` ```python # Old results = segmentor.predict(imgs=[...], mode="patch", ioconfig=config) # New results = segmentor.run(images=[...], patch_mode=True, ioconfig=config) ``` ```python # Old results = segmentor.predict(imgs=[...], mode="wsi", ioconfig=config) # New results = segmentor.run(images=[...], patch_mode=False, ioconfig=config) ``` #### Use the new I/O configs ```python from tiatoolbox.models.engine.io_config import IOSegmentorConfig config = IOSegmentorConfig( patch_input_shape=(256, 256), stride_shape=(240, 240), input_resolutions=[{"resolution": 0.25, "units": "mpp"}], save_resolution={"units": "baseline", "resolution": 1.0} ) ``` #### Specify the output format ```python results = segmentor.run( images=[...], ioconfig=ioconfig, output_type="zarr", # or "dict", "annotationstore", "qupath" save_dir="outputs/" ) ``` #### Update imports - `tiatoolbox.typing` → `tiatoolbox.type_hints` #### Install requirements - Python **3.10+** required - On Windows: install OpenSlide via `pip install openslide-bin` **Full Changelog:** v1.6.0...v2.0.0 --------- Signed-off-by: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Co-authored-by: measty <20169086+measty@users.noreply.github.com> Co-authored-by: Jiaqi-Lv <60471431+Jiaqi-Lv@users.noreply.github.com> Co-authored-by: adamshephard <39619155+adamshephard@users.noreply.github.com> Co-authored-by: Mostafa Jahanifar <74412979+mostafajahanifar@users.noreply.github.com> Co-authored-by: John Pocock <John-P@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Yijie Zhu <120978607+YijieZhu15@users.noreply.github.com> Co-authored-by: Aleksandar Acic <32873451+aacic@users.noreply.github.com> Co-authored-by: Abdol A <u2271662@live.warwick.ac.uk> Co-authored-by: Abishek <abishekraj6797@gmail.com> Co-authored-by: behnazelhaminia <30952176+behnazelhaminia@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Adam Shephard <adam.shephard@warwick.ac.uk> Co-authored-by: gozdeg <gozdegunesli@gmail.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: mbasheer04 <78800844+mbasheer04@users.noreply.github.com> Co-authored-by: vqdang <24943262+vqdang@users.noreply.github.com>

@aacic

🔖 Release 2.0.0 (#1031) ## TIAToolbox v2.0.0 (2026-03-11) ### ✨ Major Updates and Feature Improvements #### ⚙️ Engine Redesign (PR #578) TIAToolbox 2.0.0 introduces a completely re-engineered inference engine designed for significant performance, scalability, and memory-efficiency improvements. #### Key Enhancements - A modern processing stack built on **Dask** (parallel/distributed execution) and **Zarr** (chunked, out-of-core storage) - **Standardised output formats** across all engines: - Python `dict` - **Zarr** - **AnnotationStore** (SQLite-backed) - **QuPath JSON** - Cleaner runtime behavior with reduced warning noise and a unified progress bar - More predictable memory usage through chunked streaming - Broader test coverage across engine components ### 🗺️ Improved QuPath Support Enhancements include: - Better handling of **GeoJSON** - Support for **multipoint geometries** (#841) - Improved semantic output helpers: - `dict_to_store_semantic_segmentor` (#926) - OME-TIFF probability overlays (#929) ### 🔬 New Nucleus Detection Engine A dedicated nucleus detection pipeline has been added, built on the redesigned engine for improved accuracy and efficient large-scale processing. #### 🧠 KongNet Model Family TIAToolbox 2.0.0 introduces **KongNet**, a high-performance architecture that achieved top results across multiple international challenges: - 🥇 **1st place: MONKEY Challenge (overall detection)** - 🥇 **1st place: MIDOG (mitosis detection)** - ⭐ Top-tier performance on **PUMA** Multiple pretrained variants are available (CoNIC, PanNuke, MONKEY, PUMA, MIDOG), each with standardised IO configurations. ### 🧬 Expanded Foundation Model Support Additional foundation models are now supported (#906), broadening the range of high-capacity architectures available for feature extraction and downstream tasks. ### 🖼️ SAM Segmentation in TIAViz TIAViz now integrates Meta’s Segment Anything Model (SAM), enabling: - Interactive segmentation - Rapid region extraction - Exploratory annotation workflows Simplified SAM usage (#968) streamlines its integration into analysis pipelines. ### 🧩 Enhanced WSIReader & Metadata Handling Major improvements include: - More robust cross-vendor **metadata extraction** (#1001) - **Multichannel image support** (PR #825) for immunofluorescence and non-RGB modalities - Simplified Windows installation using `openslide-bin` (no manual DLL steps) - macOS Tileserver fix (#976) - Improved DICOM reading (#934) ### ☁️ New Cloud-Native Reader: FsspecJSONWSIReader (PR #897) A new reader supporting **fsspec-compatible filesystems**, enabling seamless access to WSIs stored on: - S3 - GCS - Azure - HPC clusters - Any fsspec-supported backend This enables cloud-native and distributed data workflows. Contributed by @aacic ### 🤗 Pretrained Models Migrated to Hugging Face All pretrained models and sample assets have been migrated (#945, #983), improving: - Download reliability - Versioning and reproducibility - Caching and CI integration - Licensing clarity per model family ### 🛡️ Security, Compatibility & Tooling #### 🔐 Security & Dependency Updates - Dependency upgrades - Internal security improvements - Explicit workflow permissions added (#1021, #1023) #### 🐍 Python Version Support - **Dropped:** Python **3.9** - **Added:** Python **3.13** - **Supported:** Python 3.10–3.13 - Updated CUDA wheel source to **cu126** #### 🛠️ Developer Tooling & CI/CD - Expanded **mypy** type-checking coverage (#912, #931, #935, #951) - Updated pre-commit hooks and general formatting - CI uses **CPU-only PyTorch** for faster, more reliable builds (#974, #979) - Updated pip install workflow (#1013) - Added new **Python 3.13 Docker images** (#1014, #1019) ### 🧹 Bug Fixes & Stability Improvements - Fixed multi-GPU behaviour with `torch.compile` (#923) - Fixed DICOM reading issue (#934) - Fixed annotation contour handling with holes (#956) - Fixed consecutive annotation load bug (#927) - Fixed SCCNN model issues (#970) - Fixed MapDe `dist_filter` shape issue (#914) - Improved notebook reliability on Colab (#1026–#1030) - macOS TileServer issues resolved (#976) ### 🧭 Migration Guide for Users #### 🔄 Updating from 1.x to 2.0.0 #### Update calls: replace `.predict()` with `.run()` ```python # Old results = segmentor.predict(imgs=[...], ioconfig=config) # New results = segmentor.run(images=[...], ioconfig=config) ``` #### Use `patch_mode`: replace `mode="patch"` with `patch_mode=True` and `mode="tile"` or "wsi" with `patch_mode=False` ```python # Old results = segmentor.predict(imgs=[...], mode="patch", ioconfig=config) # New results = segmentor.run(images=[...], patch_mode=True, ioconfig=config) ``` ```python # Old results = segmentor.predict(imgs=[...], mode="wsi", ioconfig=config) # New results = segmentor.run(images=[...], patch_mode=False, ioconfig=config) ``` #### Use the new I/O configs ```python from tiatoolbox.models.engine.io_config import IOSegmentorConfig config = IOSegmentorConfig( patch_input_shape=(256, 256), stride_shape=(240, 240), input_resolutions=[{"resolution": 0.25, "units": "mpp"}], save_resolution={"units": "baseline", "resolution": 1.0} ) ``` #### Specify the output format ```python results = segmentor.run( images=[...], ioconfig=ioconfig, output_type="zarr", # or "dict", "annotationstore", "qupath" save_dir="outputs/" ) ``` #### Update imports - `tiatoolbox.typing` → `tiatoolbox.type_hints` #### Install requirements - Python **3.10+** required - On Windows: install OpenSlide via `pip install openslide-bin` **Full Changelog:** v1.6.0...v2.0.0 --------- Signed-off-by: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Co-authored-by: measty <20169086+measty@users.noreply.github.com> Co-authored-by: Jiaqi-Lv <60471431+Jiaqi-Lv@users.noreply.github.com> Co-authored-by: adamshephard <39619155+adamshephard@users.noreply.github.com> Co-authored-by: Mostafa Jahanifar <74412979+mostafajahanifar@users.noreply.github.com> Co-authored-by: John Pocock <John-P@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Yijie Zhu <120978607+YijieZhu15@users.noreply.github.com> Co-authored-by: Aleksandar Acic <32873451+aacic@users.noreply.github.com> Co-authored-by: Abdol A <u2271662@live.warwick.ac.uk> Co-authored-by: Abishek <abishekraj6797@gmail.com> Co-authored-by: behnazelhaminia <30952176+behnazelhaminia@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Adam Shephard <adam.shephard@warwick.ac.uk> Co-authored-by: gozdeg <gozdegunesli@gmail.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: mbasheer04 <78800844+mbasheer04@users.noreply.github.com> Co-authored-by: vqdang <24943262+vqdang@users.noreply.github.com>

shaneahmed added 3 commits March 24, 2023 11:18

🔧 Use pyproject.toml for bdist_wheel configuration

ef55e95

- Use `pyproject.toml` for `bdist_wheel` configuration

Merge remote-tracking branch 'origin/develop' into dev-define-engines…

49a0624

…-abc

⚡ Improve Engines performance and implementation

8ba6def

- Improve `Engines` performance and implementation

shaneahmed self-assigned this Mar 31, 2023

shaneahmed added the enhancement New feature or request label Mar 31, 2023

Merge branch 'develop' into dev-define-engines-abc

5cbcfcf

shaneahmed added this to the Release v2.0.0 milestone Apr 10, 2023

shaneahmed mentioned this pull request Apr 19, 2023

🩹 Support for np.ndarray and WSIReader in PatchPredictor #576

Closed

♻️ Refactor engines_abc.py

fac1000

- Refactor engines_abc.py

shaneahmed changed the title ~~⚡ Improve Engines Performance and Implementation~~ ⚡ Improve Engine Performance and Implementation Apr 28, 2023

shaneahmed added 9 commits May 5, 2023 22:17

Merge branch 'develop' into dev-define-engines-abc

a72d9ba

Merge branch 'develop' into dev-define-engines-abc

57ea44a

Merge branch 'develop' into dev-define-engines-abc

6618161

Merge branch 'develop' into dev-define-engines-abc

6996764

Merge branch 'develop' into dev-define-engines-abc

3584f6c

Merge branch 'develop' into dev-define-engines-abc

eada692

Merge branch 'develop' into dev-define-engines-abc

77f1992

Merge branch 'develop' into dev-define-engines-abc

a477d32

Merge branch 'develop' into dev-define-engines-abc

f3e33b9

shaneahmed linked an issue Jul 14, 2023 that may be closed by this pull request

Shifted patches when merging patch predictions! #634

Closed

shaneahmed mentioned this pull request Jul 14, 2023

Shifted patches when merging patch predictions! #634

Closed

shaneahmed and others added 8 commits July 21, 2023 17:17

Merge branch 'develop' into dev-define-engines-abc

7d35285

Merge branch 'develop' into dev-define-engines-abc

7bad284

[pre-commit.ci] auto fixes from pre-commit.com hooks

36fd629

for more information, see https://pre-commit.ci

Merge branch 'develop' into dev-define-engines-abc

443141c

Merge branch 'develop' into dev-define-engines-abc

b9d8c38

[pre-commit.ci] auto fixes from pre-commit.com hooks

e608f7b

for more information, see https://pre-commit.ci

Merge branch 'develop' into dev-define-engines-abc

1d7f5c0

[pre-commit.ci] auto fixes from pre-commit.com hooks

b956bf5

for more information, see https://pre-commit.ci

Jiaqi-Lv and others added 6 commits February 20, 2026 16:22

✅ Add Tests for coverage

c4e2be7

📝 Update notebook 07-advanced-modeling.ipynb (#1003)

76a8026

- Update Jupyter Notebook `07-advanced-modeling.ipynb`

shaneahmed marked this pull request as ready for review February 24, 2026 16:37

shaneahmed and others added 9 commits February 24, 2026 16:59

[skip ci] 📝 Update notebooks

6ea553d

📝 Update docstrings for patch_predictor.py

46e9d1b

📝 Update idars Notebook for the New API (#1005)

4d990ed

- Update `idars` notebook and clean outputs in multitask segmentor and nuclei instance segmentor --------- Co-authored-by: Jiaqi Lv <lvjiaqi9@gmail.com>

[skip ci] 📝 Update slide-graph.ipynb (#1008)

1676db8

- Update slide-graph notebooks

Merge branch 'develop' into dev-define-engines-abc

439e2b1

♻️ Refactor models.engine to models.engines for Consistency (#1009)

4ac2245

- Refactor `models.engine` to `models.engines` for Consistency

Merge branch 'develop' into dev-define-engines-abc

e30e2a0

🔥 Remove github actions from dev-define-engines-abc branch

0528eff

Jiaqi-Lv requested a review from Copilot March 4, 2026 14:55

Copilot AI reviewed Mar 4, 2026

View reviewed changes

Copilot AI mentioned this pull request Mar 4, 2026

Reply to PR summary request comment #1011

Closed

shaneahmed added 3 commits March 5, 2026 06:54

⏪ Revert Changes to API engines to engine (#1012)

b5111fd

* Revert "♻️ Refactor `models.engine` to `models.engines` for Consistency (#1009)" This reverts commit 4ac2245. * ⏪ Revert `engines` to `engine` API change

🐛 Fix tests

a3fda11

💡 Address Co-Pilot Comments

b6815f3

shaneahmed merged commit 76b59c5 into develop Mar 5, 2026
27 checks passed

shaneahmed deleted the dev-define-engines-abc branch March 10, 2026 12:09

shaneahmed mentioned this pull request Mar 11, 2026

🔖 Release 2.0.0 #1031

Merged

shaneahmed mentioned this pull request Mar 11, 2026

🔖 Release 2.0.0 #1033

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⚡ Improve `Engine` Performance and Implementation#578

⚡ Improve `Engine` Performance and Implementation#578
shaneahmed merged 218 commits intodevelopfrom
dev-define-engines-abc

shaneahmed commented Mar 31, 2023 •

edited

Loading

Uh oh!

codecov Bot commented Mar 31, 2023 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jiaqi-Lv commented Mar 4, 2026

Uh oh!

Copilot AI commented Mar 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

shaneahmed commented Mar 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's New

1. Unified Engine Architecture

2. Optimized Data Processing

3. Flexible Output Types

4. Enhanced Model Architectures

5. New Engine Implementations

6. New Pre-trained Models

7. Enhanced Configuration

8. Improved CLI Infrastructure

Key Changes

Migration Notes

Example Usage

Uh oh!

codecov Bot commented Mar 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jiaqi-Lv commented Mar 4, 2026

Uh oh!

Copilot AI commented Mar 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

shaneahmed commented Mar 31, 2023 •

edited

Loading

codecov Bot commented Mar 31, 2023 •

edited

Loading