-
Notifications
You must be signed in to change notification settings - Fork 17
Description
Summary
PR #1402 introduced two medium-severity reliability/observability regressions in partition export flows:
- Scheduler slot accounting increments even when scheduling fails.
- Local fallback for
system.replicated_partition_exportscan omit non-pending exports.
This issue tracks both fixes as follow-up work.
Affected Area
- Partition export scheduler and task dispatch
- Partition export updater local cache path
system.replicated_partition_exportslocal fallback behavior
1) Scheduler slot accounting increments after failed schedule attempt
Impact
Available background slots may be underutilized within a scheduler cycle, delaying export progress under failures.
Where
src/Storages/MergeTree/ExportPartitionTaskScheduler.cpp (run(), non-lock_inside_the_task branch)
Current behavior
When storage.exportPartToTable(...) throws, scheduled_exports_count is still incremented, consuming slot budget even though no task was actually scheduled.
try
{
storage.exportPartToTable(...);
}
catch (const Exception &)
{
zk->tryRemove(fs::path(storage.zookeeper_path) / "exports" / key / "locks" / zk_part_name);
}
scheduled_exports_count++;Repro (logical)
- Have multiple pending parts and more than one available background slot.
- Force the first
exportPartToTable(...)attempt to throw. - Observe fewer successful schedules in that cycle than available slots allow.
Expected behavior
scheduled_exports_count should increment only when a part export task is successfully scheduled.
Proposed fix
Move/infer scheduled_exports_count++ so it executes only on confirmed successful scheduling.
Test direction
Add an integration/unit test with controlled first-attempt failure and assert subsequent eligible parts are still scheduled up to available capacity.
2) Local fallback can omit non-pending exports in system.replicated_partition_exports
Impact
system.replicated_partition_exports can miss completed/failed exports when local fallback path is used (for example, no MULTI_READ support), causing incomplete observability.
Where
src/Storages/MergeTree/ExportPartitionManifestUpdatingTask.cpp(poll()/ local info path)src/Storages/StorageReplicatedMergeTree.cpp(remote/local information branch)
Current behavior
Updater skips non-pending exports during polling, while local query path depends on tracked task entries.
if (!is_pending)
{
LOG_INFO(storage.log, "Skipping {}: status is not PENDING", key);
continue;
}Repro (logical)
- Run with local fallback path (no
MULTI_READor remote path not used). - Have export entries that are already non-pending.
- Query
system.replicated_partition_exports; some expected rows are missing.
Expected behavior
System table should include completed/failed exports consistently across remote and local info paths.