feat: add managed Python venv with platform-specific setup/build/validate scripts

Replace ad-hoc dependency installation with a formal virtual environment workflow:
- Add requirements.txt with pinned minimum versions for all dependencies
- Add .python-version (3.11) for pyenv/asdf integration
- Add scripts/setup.sh and setup.bat to create .venv/ with system dep auto-install
  (Homebrew on macOS, apt on Linux, winget/MSYS2 guidance on Windows)
- Add scripts/validate.sh and validate.bat with component selection (all/entities/briefs)
- Add scripts/build.sh and build.bat with component selection (all/entities/briefs/pdf)
- Remove unsafe --break-system-packages auto-install hacks from pipeline scripts
- Update CI workflow to use requirements.txt and run all pipeline steps
- Update all documentation to reference wrapper scripts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: hooman

.github/workflows/validate.yml

@@ -14,10 +14,14 @@ jobs:
           python-version: '3.11'
 
       - name: Install dependencies
-        run: pip install pyyaml jsonschema jinja2 ftfy
+        run: pip install -r requirements.txt
 
       - name: Validate schemas and cross-references
-        run: python pipeline/validate.py
+        run: |
+          python pipeline/validate.py
+          python pipeline/validate_briefs.py
 
       - name: Build outputs
-        run: python pipeline/build.py
+        run: |
+          python pipeline/build.py
+          python pipeline/build_briefs.py

.gitignore

@@ -2,6 +2,7 @@
 __pycache__/
 *.pyc
 *.pyo
+.venv/
 
 # macOS
 .DS_Store
@@ -10,7 +11,7 @@ __MACOSX/
 # Claude Code local state (worktrees, plans, settings)
 .claude/
 
-# Generated artifacts — rebuild with: python3 build.py
+# Generated artifacts — rebuild with: bash scripts/build.sh
 # Excluded to keep repo size manageable for project knowledge sync
 output/
 

.python-version

@@ -0,0 +1 @@
+3.11

docs/ARCHITECTURE.md

@@ -221,10 +221,10 @@ A4, Georgia serif, running page numbers. Brief assembly order controlled by
 CLI options:
 
 ```bash
-python pipeline/build_pdf.py                    # both tiers
-python pipeline/build_pdf.py --briefs-only      # Tier 1 only
-python pipeline/build_pdf.py --full-only        # Tier 2 only
-python pipeline/build_pdf.py --date 2026-03-04  # override release date
+bash scripts/build.sh pdf                       # both tiers
+bash scripts/build.sh pdf --briefs-only         # Tier 1 only
+bash scripts/build.sh pdf --full-only           # Tier 2 only
+bash scripts/build.sh pdf --date 2026-03-04     # override release date
 ```
 
 ---
@@ -234,13 +234,13 @@ python pipeline/build_pdf.py --date 2026-03-04  # override release date
 ```bash
 # 1. Edit YAML source files in data/
 # 2. Validate
-python pipeline/validate.py && python pipeline/validate_briefs.py
+bash scripts/validate.sh
 
 # 3. Build markdown
-python pipeline/build.py && python pipeline/build_briefs.py
+bash scripts/build.sh
 
 # 4. Build PDFs
-python pipeline/build_pdf.py
+bash scripts/build.sh pdf
 
 # 5. Commit source changes only (output/ and releases/ are gitignored)
 git add data/ schemas/ templates/ pipeline/ scripts/ docs/ *.md

docs/CLAUDE_CODE_INSTRUCTIONS.md

@@ -94,25 +94,42 @@ framework/
 
 ---
 
+## Environment Setup
+
+Requires Python 3.9+ (3.11+ recommended). Run once to create the virtual environment:
+
+```bash
+# macOS / Linux
+bash scripts/setup.sh
+
+# Windows
+scripts\setup.bat
+```
+
+This creates `.venv/` with all dependencies from `requirements.txt`.
+
+---
+
 ## Core Rules
 
 ### Rule 1: Never edit output/*.md files
 These are generated artifacts. Edit the YAML source, then rebuild.
 
 ### Rule 2: Validate before every commit
 ```bash
-python pipeline/validate.py              # entities + content modules
-python pipeline/validate_briefs.py       # briefs
-python pipeline/validate.py variables    # single entity type
+bash scripts/validate.sh                # everything (entities + briefs)
+bash scripts/validate.sh entities       # entities + content modules only
+bash scripts/validate.sh briefs         # briefs only
+bash scripts/validate.sh variables      # single entity type
 ```
 
 ### Rule 3: Build after every data change
 ```bash
-python pipeline/build.py                 # entity reports + content modules
-python pipeline/build_briefs.py          # convergence briefs
-python pipeline/build.py variables       # one report only
-python pipeline/build.py content         # content modules only
-python pipeline/build.py --validate      # validate then build
+bash scripts/build.sh                   # everything (entities + briefs)
+bash scripts/build.sh entities          # entity reports + content modules
+bash scripts/build.sh briefs            # convergence briefs
+bash scripts/build.sh pdf               # PDF releases
+bash scripts/build.sh --validate        # validate then build all
 ```
 
 ### Rule 4: Atomic updates
@@ -415,11 +432,11 @@ When a Claude session produces an Integration Spec, translate it to database ope
 6. **Version bump** → update `version`, `date`, `source` metadata fields
 7. **Validate:**
    ```bash
-   python pipeline/validate.py && python pipeline/validate_briefs.py
+   bash scripts/validate.sh
    ```
 8. **Build:**
    ```bash
-   python pipeline/build.py && python pipeline/build_briefs.py
+   bash scripts/build.sh
    ```
 9. **Commit** (output/ and releases/ are gitignored — do not use `git add -A`):
    ```bash
@@ -434,10 +451,10 @@ When a Claude session produces an Integration Spec, translate it to database ope
 After data integration and build:
 
 ```bash
-python pipeline/build_pdf.py                    # both tiers → releases/
-python pipeline/build_pdf.py --briefs-only      # Tier 1 only
-python pipeline/build_pdf.py --full-only        # Tier 2 only
-python pipeline/build_pdf.py --date 2026-03-05  # override release date
+bash scripts/build.sh pdf                       # both tiers → releases/
+bash scripts/build.sh pdf --briefs-only         # Tier 1 only
+bash scripts/build.sh pdf --full-only           # Tier 2 only
+bash scripts/build.sh pdf --date 2026-03-05     # override release date
 ```
 
 Then create a GitHub Release tagged `v{YYYY-MM-DD}` and attach the PDFs from
@@ -524,8 +541,8 @@ specified fields. Fields not listed are left unchanged.
    a. Read files from `staging/session_N/`
    b. Full files → copy to `data/` target path
    c. Patch files → merge field updates into existing YAML
-   d. Validate: `python pipeline/validate.py && python pipeline/validate_briefs.py`
-   e. Build: `python pipeline/build.py && python pipeline/build_briefs.py`
+   d. Validate: `bash scripts/validate.sh`
+   e. Build: `bash scripts/build.sh`
 3. Delete the consumed `staging/session_N/` directory
 4. Commit everything atomically:
    ```bash

docs/GUIDE_ENGINEERS.md

@@ -131,14 +131,18 @@ Add this to your LLM project configuration:
 ### Quick Start
 
 ```bash
-pip install pyyaml jsonschema jinja2 ftfy weasyprint markdown
-
-python pipeline/validate.py && python pipeline/validate_briefs.py   # validate all
-python pipeline/build.py && python pipeline/build_briefs.py          # build markdown
-python pipeline/build_pdf.py                                        # build PDF releases
+# One-time setup (creates .venv/ with all dependencies)
+bash scripts/setup.sh          # macOS / Linux
+scripts\setup.bat              # Windows
+
+# Run the pipeline
+bash scripts/validate.sh       # validate all
+bash scripts/build.sh          # build markdown
+bash scripts/build.sh pdf      # build PDF releases
 ```
 
-All commands run from the repository root.
+All commands run from the repository root. The wrapper scripts handle
+virtual environment activation automatically.
 
 ### What Each Script Does
 

pipeline/README.md

@@ -10,20 +10,53 @@ Build and validation scripts for the ITP analytical database.
 | `build_briefs.py` | Render brief YAML to markdown |
 | `build_pdf.py` | Convert rendered markdown to PDF releases |
 
+## Setup
+
+Requires Python 3.9+ (3.11+ recommended). Run the setup script once to create a virtual environment:
+
+```bash
+# macOS / Linux
+bash scripts/setup.sh
+
+# Windows
+scripts\setup.bat
+```
+
+This creates `.venv/` with all dependencies from `requirements.txt`.
+
 ## Usage
 
-Run from the **repository root**:
+Use the wrapper scripts (they activate the virtual environment automatically):
 
 ```bash
-python pipeline/validate.py          # validate all entities
-python pipeline/validate.py modules  # validate one entity type
-python pipeline/validate_briefs.py   # validate briefs
-python pipeline/build.py             # build all output
-python pipeline/build_pdf.py        # generate PDF releases
+# macOS / Linux
+bash scripts/validate.sh              # validate everything
+bash scripts/validate.sh entities     # entities only
+bash scripts/validate.sh briefs       # briefs only
+bash scripts/validate.sh variables    # single entity type
+
+bash scripts/build.sh                 # build everything
+bash scripts/build.sh briefs          # briefs only
+bash scripts/build.sh pdf             # PDF releases
+bash scripts/build.sh --validate      # validate then build
+
+# Windows
+scripts\validate.bat
+scripts\build.bat
 ```
 
-## Dependencies
+Or run pipeline scripts directly (with the virtual environment activated):
 
 ```bash
-pip install pyyaml jsonschema jinja2 ftfy weasyprint
+source .venv/bin/activate             # macOS / Linux
+python pipeline/validate.py
+python pipeline/build.py
+deactivate
 ```
+
+## Dependencies
+
+Managed via `requirements.txt` in the repository root. Core pipeline needs
+`pyyaml`, `jsonschema`, `jinja2`, `ftfy`. PDF generation additionally needs
+`weasyprint` and `markdown` (plus system libraries — see `requirements.txt`
+for platform-specific install notes).

pipeline/build_briefs.py

@@ -5,21 +5,8 @@
 import sys
 from pathlib import Path
 
-try:
-    import yaml
-except ImportError:
-    import subprocess
-    subprocess.check_call([sys.executable, "-m", "pip", "install",
-                           "pyyaml", "--break-system-packages", "-q"])
-    import yaml
-
-try:
-    import jinja2
-except ImportError:
-    import subprocess
-    subprocess.check_call([sys.executable, "-m", "pip", "install",
-                           "jinja2", "--break-system-packages", "-q"])
-    import jinja2
+import yaml
+import jinja2
 
 
 BASE = Path(__file__).resolve().parent.parent

pipeline/validate_briefs.py

@@ -5,22 +5,8 @@
 import sys
 from pathlib import Path
 
-# jsonschema may need install
-try:
-    import jsonschema
-except ImportError:
-    import subprocess
-    subprocess.check_call([sys.executable, "-m", "pip", "install",
-                           "jsonschema", "--break-system-packages", "-q"])
-    import jsonschema
-
-try:
-    import yaml
-except ImportError:
-    import subprocess
-    subprocess.check_call([sys.executable, "-m", "pip", "install",
-                           "pyyaml", "--break-system-packages", "-q"])
-    import yaml
+import jsonschema
+import yaml
 
 
 BASE = Path(__file__).resolve().parent.parent

requirements.txt

@@ -0,0 +1,16 @@
+# ITP Framework — Python dependencies
+# Install into a virtual environment: pip install -r requirements.txt
+#
+# Core pipeline (validate + build)
+pyyaml>=6.0.1
+jsonschema>=4.21.0
+jinja2>=3.1.3
+ftfy>=6.2.0
+
+# PDF generation (optional — only needed for `build_pdf.py`)
+# WeasyPrint requires system libraries: Cairo, Pango, GDK-PixBuf.
+# macOS:   brew install cairo pango gdk-pixbuf libffi
+# Ubuntu:  apt install libcairo2 libpango-1.0-0 libgdk-pixbuf2.0-0 libffi-dev
+# Windows: See https://doc.courtbouillon.org/weasyprint/stable/first_steps.html
+weasyprint>=62.0
+markdown>=3.6