Skip to content

Natural language custom review rules #12

@haasonsaas

Description

@haasonsaas

Problem

DiffScope supports review rules in YAML format with structured fields (scope, severity, category). All tier-1 competitors additionally accept rules written in plain English prose:

"Never allow direct SQL string concatenation — always use parameterized queries."
"All public API endpoints must have rate limiting middleware."
"React components should not call hooks conditionally."

This dramatically lowers the barrier for teams to codify their conventions without learning a YAML schema.

Proposed Solution

  1. Accept natural language rules alongside existing YAML rules:

    rules:
      - "All database queries must use parameterized statements"
      - "Public functions must have doc comments"
      - source: ./rules/security.yml  # existing YAML format still works
  2. Inject prose rules directly into the system prompt as review instructions

  3. Support a .diffscope-rules plaintext file — one rule per line, for teams that don't want YAML

  4. Map prose rules to structured findings in the output (category, severity inferred by the LLM)

  5. Allow combining prose rules with YAML rules — YAML for precision, prose for conventions

Competitive Context

  • CodeRabbit: Natural language rules in .coderabbit.yaml instructions field
  • Qodo Merge: Extra instructions in natural language
  • Ellipsis: Full natural language style guide enforcement
  • Greptile: Natural language rule definitions

Priority

Tier 2 — Strong differentiator. Lowers adoption barrier significantly for teams.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions