syntax-color-objectscript

Workspace for syntax coloring with a clean split between:

• syntax analysis (highlight-spans)
• style selection (theme-engine)

This lets one parser/theme pipeline support multiple outputs:

• ANSI/VT terminal rendering
• native C paint engines
• future GUI/web adapters

What This Accomplishes

This project turns source code into style-ready data in two stages:

1. Parse and classify code tokens into semantic capture names (keyword, comment, number, etc.).
2. Map those names to concrete visual styles (RGB + bold/italic/underline).

The key benefit is decoupling parser logic from theme logic, so each can evolve independently.

Workspace Layout

crates/
  highlight-spans/  # Tree-sitter -> spans + attr table
  theme-engine/     # capture name -> style resolution + built-in theme loader
    themes/         # built-in JSON themes (tokyonight/solarized)
  render-ansi/      # styled ranges -> ANSI/VT escape output
  theme-engine-ffi/ # C ABI wrapper around theme-engine
docs/
  architecture.md
  highlight-spans.md
  theme-engine.md
  render-ansi.md
  integration.md

Crates

highlight-spans

Purpose:

• Convert source text (ObjectScript/SQL/Python/Markdown/MDX) into (attr_id, start_byte, end_byte) spans.
• Return an attribute table mapping attr_id -> capture_name.
• Treat mdx as a temporary alias to SQL highlighting (for InterSystems MDX content).
• Crate README

Depends on:

• tree-sitter-objectscript-playground = "1.5.1"
• tree-sitter-python = "0.25.0"
• tree-sitter-md = "0.5.3"
• tree-sitter-highlight = ">=0.26.6"
• tree-sitter = ">=0.26.6"
• bundled SQL grammar from DerekStride/tree-sitter-sql (vendor/tree-sitter-sql/src/*, vendor/tree-sitter-sql/queries/highlights.scm)

theme-engine

Purpose:

• Resolve capture names to concrete styles:
  • fg/bg RGB
  • bold, italic, underline
• Resolve UI-role styles (statusline, tab_active, selection, etc.).
• Provide theme default terminal fg/bg for OSC 10/11 integration.
• Normalize capture keys (@comment and comment both resolve).
• Support fallback (comment.documentation -> comment -> normal).
• Include built-in themes: tokyonight-dark, tokyonight-moon, tokyonight-light, tokyonight-day, solarized-dark, solarized-light.
• Crate README

render-ansi

Purpose:

• Convert highlighted byte spans into ANSI/VT escaped text.
• Provide line-oriented APIs (Vec<String>) for terminal rendering.
• Provide incremental VT patching with configurable terminal origin offsets (row, col).
• Provide bridge-friendly auto mode selection (vt_patch_bridge) when origin is omitted.
• Compute incremental patch columns using grapheme/display-width logic (wide Unicode and tabs).
• Provide OSC helpers for terminal default fg/bg updates from theme values.
• Keep renderer logic separate from parsing and theme selection.
• Crate README

theme-engine-ffi

Purpose:

• Expose theme-engine APIs to C hosts via a stable FFI layer.
• Resolve both syntax capture styles and UI role styles from C.
• Provide theme default terminal fg/bg colors for OSC 10/11 integration.
• Crate README

Data Flow

source code
  -> highlight-spans
     -> attrs: [{id, capture_name}]
     -> spans: [{attr_id, start_byte, end_byte}]
  -> theme-engine
     -> style per capture_name
  -> renderer (ANSI or C painter)

Quick Example

use highlight_spans::{Grammar, SpanHighlighter};
use theme_engine::Theme;

let mut highlighter = SpanHighlighter::new()?;
let result = highlighter.highlight(b"set x = 42", Grammar::ObjectScript)?;

let theme = Theme::from_json_str(r#"{
  "styles": {
    "normal": { "fg": { "r": 220, "g": 220, "b": 220 } },
    "number": { "fg": { "r": 255, "g": 180, "b": 120 } }
  }
}"#)?;

for span in &result.spans {
    let capture = &result.attrs[span.attr_id].capture_name;
    let style = theme.resolve(capture);
    // renderer applies `style` to source[span.start_byte..span.end_byte]
}
# Ok::<(), Box<dyn std::error::Error>>(())

Documentation

• arc42 Architecture
• C4 Model
• Design Doc
• Integration Steps
• Incremental Terminal Highlighting Guide
• Release and Publish Guide
• Usage Examples
• Data Structure: HighlightResult
• Data Structure: Theme
• Data Structure: StyledSpan
• Code Health Report
• Edge-Case Test Plan

Test

cargo test