refactor(delphi): Remove legacy poller/system architecture and dead code by jucor · Pull Request #2343 · compdemocracy/polis

jucor · 2026-01-07T15:14:36Z

Here's some clean up of dead code: using Claude Code Opus 4.5 to clean up behind Claude Code Sonnet 3.7 from last year ;-)
There are probably still code paths that are unused, and I have not removed the markdown files created during the python port last year -- they'll still serve until we're fully sure of the codebase, I think.

Summary

Systematic cleanup of dead/unused code from the delphi/ folder (ported from Clojure ~1 year ago). Removed 730 net lines of dead code across 6 commits, with all tests passing.

Key Changes

Removed (1,162 lines)

✅ Legacy poller/system architecture:
- polismath/__main__.py (150 lines) - Legacy entry point
- polismath/system.py (208 lines) - Legacy system manager
- polismath/poller.py (507 lines) - PostgreSQL continuous polling
- polismath/components/server.py (297 lines) - Legacy FastAPI server

Removed (6 unused imports)

squareform from pca_kmeans_rep/corr.py
weighted_mean, weighted_means from pca_kmeans_rep/clusters.py
JSON, QueuePool from database/postgres.py and scripts/job_poller.py

Documentation

Updated RUNNING_THE_SYSTEM.md (removed SystemManager references)
All markdown files kept in docs/ (not archived)
Added comprehensive cleanup report

⚠️ False Positive (Corrected)

Initially deleted polismath/utils/general.py (263 lines) due to faulty grep verification, but this file had 4 active imports in production code. It has been restored and tests now pass.

Root cause: Grep was likely run from wrong directory. Network issues prevented running tests before initial commit.

Verification

Tests Pass ✅

211 passed, 7 skipped, 2 xfailed

Legacy System Gone

grep -r "from polismath.system\|from polismath.poller" --include="*.py" .
# No matches (except in archived docs and cleanup report)

Entry Points Preserved

Current Delphi entry point (run_delphi:main) and all other entry points remain functional:

run-delphi (Docker/Makefile main entry point)
run-math-pipeline
run-umap-pipeline
calculate-extremity
calculate-priorities
reset-conversation
create-datamapplot

Note: The delphi CLI script is old and not fully working; Docker uses run_delphi.

Impact

12 files changed, 497 insertions(+), 1201 deletions(-)
Net: -704 lines

Future Work

Vulture analysis identified 92 additional items (mostly 60% confidence) for future cleanup:

12 high-confidence (90%+) unused imports (~12 lines, low risk)
Various potentially unused classes/functions requiring domain expert review

Documentation

Full details in delphi/docs/DEAD_CODE_CLEANUP_REPORT.md including:

Methodology (Vulture static analysis + grep verification)
Complete commit timeline
Lessons learned and process improvements
Reproducibility instructions

Commits: cebd9d2b6 through 645894d38
Base: edge
Tests: ✅ All passing

Remove dead code that was replaced by scripts/job_poller.py + DynamoDB job queue: - polismath/__main__.py (legacy entry point) - polismath/system.py (legacy System/SystemManager) - polismath/poller.py (legacy Poller/PollerManager) - polismath/components/server.py (legacy Server/ServerManager) Updated __init__.py files to remove imports of deleted modules. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Remove polismath/utils/general.py (unused utility functions - 263 lines) - Remove unused squareform import from corr.py - Archive outdated docs to docs/archive/: - conversion_plan.md (conversion complete) - NEXT_STEPS.md (outdated) - project_structure.md (proposed, not actual)

- Update RUNNING_THE_SYSTEM.md: remove SystemManager references, add current CLI usage - Archive architecture_overview.md (describes Clojure, not Python) - Archive summary.md (references deleted poller/server/system components) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Detailed documentation of the dead code identification and removal process, including actual vulture output (98 findings) and recommendations for future cleanup. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

This commit corrects a critical false positive from the dead code cleanup: 1. RESTORED: polismath/utils/general.py (263 lines) - Was incorrectly marked as dead code - Actually has 4 active imports in production code: * postgres.py: postgres_vote_to_delphi() * run_math_pipeline.py: postgres_vote_to_delphi() * repness.py: AGREE, DISAGREE constants * clusters.py: (import existed but unused - see below) 2. REMOVED: Unused imports (5 total) - clusters.py: weighted_mean, weighted_means - postgres.py: JSON, QueuePool - job_poller.py: JSON, QueuePool 3. ADDED: docs/DEAD_CODE_CLEANUP_REPORT_CORRECTED.md - Documents the false positive - Provides root cause analysis - Recommends process improvements Tests verified: 211 passed, 7 skipped, 2 xfailed 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Merge DEAD_CODE_CLEANUP_REPORT.md and DEAD_CODE_CLEANUP_REPORT_CORRECTED.md into a single comprehensive report documenting the entire cleanup series. Changes: - Combined both reports into unified DEAD_CODE_CLEANUP_REPORT.md - Documents false positive (general.py) and correction - Clarifies entry point is run_delphi (not delphi CLI) - Shows net impact: -730 lines from edge branch - All documentation files kept in docs/ (not archived) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Add complete output from Vulture dead code analyzer (147 findings): - 90%+ confidence: ~15 unused imports - 60%+ confidence: ~132 potentially unused functions/classes/methods This provides the raw data referenced in DEAD_CODE_CLEANUP_REPORT.md for future dead code cleanup efforts. Command used: vulture . --min-confidence 60 --exclude ".git,__pycache__,*.pyc,.venv,tests" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Remove all 90%+ confidence unused imports identified by Vulture: Files modified (10): - polismath/benchmarks/bench_repness.py (3 imports) - polismath/components/config.py (Set) - polismath/conversation/conversation.py (Set) - polismath/conversation/manager.py (Set) - polismath/database/postgres.py (Set) - polismath/utils/general.py (itertools, Set) - scripts/delphi_cli.py (Text, rprint) - umap_narrative/801_narrative_report_batch.py (csv, io) - umap_narrative/polismath_commentgraph/core/clustering.py (hdbscan, delayed, Parallel) - umap_narrative/polismath_commentgraph/utils/storage.py (QueuePool) All imports verified as unused by: 1. Vulture static analysis (90% confidence) 2. Manual grep verification 3. Test suite (45 tests passed) This addresses the high-priority cleanup items from vulture_analysis_output.txt. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Copilot

Pull request overview

This pull request performs a systematic cleanup of dead code from the delphi/ folder, which was ported from Clojure to Python approximately one year ago. The cleanup removes legacy polling/system architecture that has been replaced with a DynamoDB-based job queue system, along with various unused imports identified through Vulture static analysis. The PR is well-documented with a comprehensive cleanup report and all tests pass (211 passed, 7 skipped, 2 xfailed).

Changes:

Removed entire legacy poller/system architecture (4 files, 1,162 lines) including polismath/__main__.py, polismath/system.py, polismath/poller.py, and polismath/components/server.py
Cleaned up 6 unused imports across 11 files based on high-confidence (90%+) Vulture findings
Updated documentation to remove references to deleted components and added comprehensive cleanup report with Vulture analysis output for future reference

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
delphi/polismath/main.py	Removed legacy entry point for `python -m polismath` (no longer used)
delphi/polismath/system.py	Removed legacy System/SystemManager classes (only used by deleted main.py)
delphi/polismath/poller.py	Removed legacy PostgreSQL polling system (replaced by DynamoDB job queue)
delphi/polismath/components/server.py	Removed legacy FastAPI server component (only used by deleted system.py)
delphi/polismath/init.py	Removed exports of deleted System/SystemManager classes
delphi/polismath/components/init.py	Removed exports of deleted Server/ServerManager classes
delphi/polismath/utils/general.py	Removed unused `itertools` and `Set` imports
delphi/polismath/pca_kmeans_rep/corr.py	Removed unused `squareform` import
delphi/polismath/pca_kmeans_rep/clusters.py	Removed unused `weighted_mean`, `weighted_means` imports
delphi/polismath/database/postgres.py	Removed unused `JSON`, `Set` imports (only `JSONB` is used)
delphi/polismath/components/config.py	Removed unused `Set` import
delphi/polismath/conversation/manager.py	Removed unused `Set` import
delphi/polismath/conversation/conversation.py	Removed unused `Set` import
delphi/polismath/benchmarks/bench_repness.py	Removed unused function imports (kept _df variants that are actually used)
delphi/scripts/job_poller.py	Removed unused `JSON`, `QueuePool` imports
delphi/scripts/delphi_cli.py	Removed unused `Text`, `rprint` imports from rich library
delphi/umap_narrative/polismath_commentgraph/utils/storage.py	Removed unused `QueuePool` import
delphi/umap_narrative/polismath_commentgraph/core/clustering.py	Removed unused `hdbscan`, `joblib.Parallel`, `joblib.delayed` imports
delphi/umap_narrative/801_narrative_report_batch.py	Removed unused `csv`, `io` imports
delphi/docs/RUNNING_THE_SYSTEM.md	Updated to remove SystemManager references and modernize usage examples
delphi/docs/DEAD_CODE_CLEANUP_REPORT.md	Added comprehensive cleanup report documenting methodology, verification, and lessons learned
delphi/docs/vulture_analysis_output.txt	Added Vulture static analysis output (92 additional findings for future cleanup)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jucor · 2026-03-05T14:17:23Z

Superseded by #2423 (moved head branch to upstream for simpler stacked PR workflow).

jucor and others added 6 commits January 5, 2026 19:23

jucor mentioned this pull request Jan 7, 2026

refactor(delphi): Remove legacy poller/system architecture and dead code jucor/polis#4

Closed

jucor marked this pull request as draft January 7, 2026 15:18

jucor and others added 2 commits January 7, 2026 15:20

jucor marked this pull request as ready for review January 7, 2026 19:30

jucor requested a review from ballPointPenguin January 8, 2026 17:34

jucor requested review from tevko February 16, 2026 18:18

jucor requested a review from Copilot February 25, 2026 14:07

Copilot started reviewing on behalf of jucor February 25, 2026 14:07 View session

Copilot AI reviewed Feb 25, 2026

View reviewed changes

jucor closed this Mar 5, 2026

jucor deleted the dead-code-cleanup branch March 5, 2026 12:41

jucor restored the dead-code-cleanup branch March 5, 2026 12:43

jucor reopened this Mar 5, 2026

jucor closed this Mar 5, 2026

jucor deleted the dead-code-cleanup branch March 5, 2026 14:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(delphi): Remove legacy poller/system architecture and dead code#2343

refactor(delphi): Remove legacy poller/system architecture and dead code#2343
jucor wants to merge 8 commits intocompdemocracy:edgefrom
jucor:dead-code-cleanup

jucor commented Jan 7, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

jucor commented Mar 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jucor commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Changes

Removed (1,162 lines)

Removed (6 unused imports)

Documentation

⚠️ False Positive (Corrected)

Verification

Tests Pass ✅

Legacy System Gone

Entry Points Preserved

Impact

Future Work

Documentation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

jucor commented Mar 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jucor commented Jan 7, 2026 •

edited

Loading