Add multiple verification levels to read_ledger.py

[Copilot]

Add Multiple Verification Levels to read_ledger.py

This PR implements granular verification levels for the read_ledger.py tool, allowing users to control the trade-off between verification thoroughness and computation cost.

Changes

Added VerificationLevel enum with five levels (ordered by increasing cost):

• NONE: Parse only, no verification
• OFFSETS: Validate offset table consistency
• HEADERS: Validate transaction headers (size, version, flags)
• MERKLE: Validate merkle tree (trust first signature via merkletree.deserialise())
• FULL: Full cryptographic verification (default, unchanged)

Updated read_ledger.py CLI:

• New --verification-level option (choices: NONE, OFFSETS, HEADERS, MERKLE, FULL)
• Deprecated --insecure-skip-verification (maps to NONE for backward compatibility)
• Contextual output showing relevant metrics per level

Updated LedgerValidator:

• Accepts verification_level parameter (defaults to FULL)
• Implements incremental validation: each level builds on previous levels
• MERKLE level properly deserializes mini-tree from first signature for isolated chunk support
• Tracks transaction_count for better reporting

Documentation: Added verification levels section to doc/audit/python_library.rst with usage examples

Testing: Added test_merkle_verification_level() to validate MERKLE verification on full ledgers and isolated chunks

Backward Compatibility

✅ Default behavior unchanged (FULL verification)
✅ --insecure-skip-verification still works (deprecated)
✅ Existing code using LedgerValidator() continues to work

Verification Levels & Isolated Chunks

• NONE, OFFSETS, HEADERS: Work with individual chunks
• MERKLE: Works with isolated chunks (trusts first signature)
• FULL: Requires complete ledger context

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Add multiple verification levels to read_ledger.py</issue_title>
<issue_description>We can detect more potential corruptions of ledger files at the cost of increased computation. Currently we have a single --insecure-skip-verification flag, which disables all verification. This is required for reading isolated files, but means we do minimal verification of the result. We could do better, and be more explicit about a "no verifiation, just try to parse" mode.

Specifically, we can: validate nothing, validate the offsets table, validate each transaction's header (size points to the next offset, version and flags are allowed values), validate the merkle tree in each signature (trust the first, confirm the mini-tree in each subsequent siganture matches), validate the signatures themselves (trust the node certs in each signature transaction, enforce invariants of reconfigurations within the seen files). I don't know if all of these are worth exposing as separate options, but since they are (roughly) ordered in increasing computation cost I think the final setting is a monotonic level rather than distinct flags.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Add multiple verification levels to read_ledger.py #7502

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[eddyashton]

@copilot Python's enum.Enum is not natively comparable, use IntEnum instead.

'>=' not supported between instances of 'VerificationLevel' and 'VerificationLevel'

[Copilot]

@copilot Python's enum.Enum is not natively comparable, use IntEnum instead.

...

Fixed in commit 564c48e. Changed VerificationLevel from Enum to IntEnum to support native comparison operations.

[Copilot]

Pull request overview

This PR implements multiple verification levels for the read_ledger.py utility, allowing users to control the trade-off between verification thoroughness and computation cost. The implementation introduces five verification levels (NONE, OFFSETS, HEADERS, MERKLE, FULL) with FULL being the default, maintaining backward compatibility with existing code.

Changes:

• Added VerificationLevel IntEnum with five levels of increasing verification rigor
• Implemented offset table and transaction header validation methods
• Implemented MERKLE verification level using merkletree.deserialise() to support isolated chunk verification
• Updated CLI to accept --verification-level parameter while maintaining backward compatibility with deprecated --insecure-skip-verification flag

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
python/src/ccf/ledger.py	Added VerificationLevel enum, validation methods, and MERKLE verification logic with proper tree deserialization
python/src/ccf/read_ledger.py	Updated CLI to support verification levels with backward compatibility and contextual output messages
tests/e2e_operations.py	Removed duplicate merkle tree validation and added dedicated test for MERKLE verification level
doc/audit/python_library.rst	Added comprehensive documentation for verification levels with usage examples

[eddyashton]

@copilot Add a description of this change to CHANGELOG.md, and rewrite this PR's description to be briefer (summarise, don't list every change).

[Copilot]

@copilot Add a description of this change to CHANGELOG.md, and rewrite this PR's description to be briefer (summarise, don't list every change).

Done in commit aef3642:

• Added CHANGELOG entry under "Added" section for 7.0.0-dev10
• Rewrote PR description to be concise summary instead of detailed checklist