Skip to content

README.md

Latest commit

 

History

History
715 lines (563 loc) · 24.9 KB

README.md

File metadata and controls

715 lines (563 loc) · 24.9 KB

TypeScript MCP Server

The enterprise-grade TypeScript implementation of the CodeSight MCP Server with AI-powered code intelligence platform. Features advanced LLM integration, 14 MCP tools including 5 AI-powered capabilities, multi-language Tree-sitter parsing, and a sophisticated NAPI-RS FFI bridge with comprehensive enterprise workflows.

Overview

This module implements the MCP protocol layer that enables AI assistants like Claude to interact with codebases through natural language queries and advanced AI-powered analysis. Phase 4.1 introduces comprehensive LLM integration with 5 new AI-powered tools that provide intelligent code review, refactoring suggestions, bug prediction, context-aware code generation, and technical debt analysis.

Architecture

┌─────────────────────────────────┐
│           AI Assistants           │
│    (Claude, GPT-4, etc.)        │
└─────────────────┬───────────────┘
                  │ MCP Protocol
┌─────────────────▼───────────────┐
│        TypeScript MCP Server     │
│  • 14 MCP Tools (9 Core + 5 AI) │
│  • Multi-Provider LLM Integration│
│  • Enterprise-grade error handling│
│  • REST API + WebSocket Support  │
│  • Unified Configuration System │
└─────────────────┬───────────────┘
                  │ NAPI-RS FFI
┌─────────────────▼───────────────┐
│          Rust Core Engine        │
│  • Multi-Language Tree-sitter   │
│  • Parallel Processing (Rayon)  │
│  • Memory-Optimized Algorithms │
│  • Production-Ready Crates      │
└─────────────────┬───────────────┘
                  │
┌─────────────────▼───────────────┐
│         AI/LLM Services         │
│  • Anthropic Claude Integration  │
│  • OpenAI GPT-4 Support         │
│  • Ollama Local Models          │
│  • Intelligent Fallback System  │
│  • Context-Aware Analysis       │
└─────────────────────────────────┘

Features

✅ Phase 4.1 AI-Powered Enterprise Implementation:

Core Intelligence Engine

  • Real Code Indexing: SQLite database with 377+ indexed entities
  • Multi-Language Support: 15+ programming languages with Tree-sitter parsers
  • Functional Search: Query intent detection with relevance scoring
  • MCP Protocol: Full compliance with 14 implemented tools (9 core + 5 AI)

AI/LLM Integration (NEW)

  • Multi-Provider Support: Anthropic Claude, OpenAI GPT-4, Ollama local models
  • Intelligent Fallback: Rule-based analysis when LLM services unavailable
  • Context-Aware Analysis: Project-aware code intelligence with pattern recognition
  • Performance Optimized: Sub-second AI responses with caching and optimization

Enterprise Infrastructure

  • CLI Tools: Working index, search, stats, and AI analysis commands
  • Contract Tests: All 14 MCP tools tested and validated
  • Integration Testing: Comprehensive test suite with AI tool validation
  • Claude Desktop Integration: Full AI-powered workflow testing
  • VS Code Integration: Complete workspace analysis with AI suggestions
  • End-to-End Workflows: Real-world AI-assisted development validation
  • FFI Bridge: Complete NAPI-RS integration with graceful fallback
  • Hybrid Architecture: Optimized performance with Rust core + TypeScript + AI services
  • Error Handling: Comprehensive error management across AI and FFI boundaries
  • Enterprise CI/CD: 7 GitHub Actions workflows with AI testing pipelines
  • Production Docker: Complete containerization with AI service dependencies
  • Professional Tooling: Unified ESLint, TypeScript configs, security scanning
  • Monitoring: Prometheus metrics, Grafana dashboards, AI performance tracking
  • Performance Optimized: 1-2 second indexing, 20-50ms search, <1s AI analysis

Available MCP Tools

🤖 AI-Powered Tools (Phase 4.1 - NEW)

  1. ai_code_review - AI-powered comprehensive code review with intelligent suggestions
  2. intelligent_refactoring - AI-powered refactoring recommendations with code transformation
  3. bug_prediction - AI-powered bug prediction and proactive risk assessment
  4. context_aware_code_generation - AI-powered context-aware code generation
  5. technical_debt_analysis - Comprehensive technical debt assessment with business impact

🔧 Core Tools (Phases 3.0-3.5)

  1. search_code - Natural language search with SQLite database integration
  2. explain_function - Function explanation with comprehensive code analysis
  3. find_references - Find all references to a symbol with cross-file analysis
  4. trace_data_flow - Trace data flow through the code with variable tracking
  5. analyze_security - Analyze code for security vulnerabilities with comprehensive checks
  6. get_api_endpoints - List all API endpoints in the codebase with HTTP methods
  7. check_complexity - Analyze code complexity metrics with detailed breakdown
  8. find_duplicates - Detect duplicate code patterns with similarity scoring
  9. suggest_refactoring - Provide refactoring suggestions with implementation guidance

🏆 Complete AI-Enhanced MCP Implementation - All 14 tools are fully functional with comprehensive AI integration and testing.

Installation & Quick Start

cd typescript-mcp
npm install
npm run build

# Build Rust FFI bridge (recommended for production performance)
cd ../rust-core && cargo build --release && cd ../typescript-mcp

# Configure AI providers (optional - see AI Configuration section)
export ANTHROPIC_API_KEY="your-anthropic-api-key"  # For Claude integration
export OPENAI_API_KEY="your-openai-api-key"        # For GPT-4 integration

# Index your codebase
node dist/cli/index.js index /path/to/your/project

# Check what was indexed
node dist/cli/index.js stats
# Output: Total entities: 377 (class: 48, function: 175, interface: 140, type: 14)

# Test natural language search
node dist/cli/index.js search "authentication functions"
# Output: Found entities with relevance scores

# Test AI-powered code review (NEW)
node dist/cli/index.js ai-review --file="src/user-service.ts" --type="comprehensive"

# Start MCP server (for Claude Desktop integration with AI features)
node dist/index.js

# Run comprehensive tests including AI tools
npm test
npm run test:contract
npm run test:ai-tools
npm run test:performance

Development

Build Commands

# Development build with watch mode
npm run dev

# Production build
npm run build

# Build with Rust FFI bindings
npm run build:full

# Hybrid build (TypeScript + Rust + AI)
npm run build:hybrid

# Test contract compliance
npm run test:contract

# Test AI-powered tools
npm run test:ai-tools

# Run comprehensive testing
npm test
npm run test:coverage
npm run test:contract
npm run test:ai-tools
npm run test:performance

# Docker development with AI services
cd .. && docker-compose -f docker-compose.dev.yml up -d

Testing

# Run all tests
npm test

# Run contract tests specifically
npm run test:contract

# Run with coverage
npm run test:coverage

# Run contract tests
npm run test:contract

# Run performance benchmarks
npm run test:performance

# Watch mode for development
npm run test:watch

Testing

# Run all tests (72/72 passing)
npm test

# Run contract tests specifically
npm run test:contract

# Run with coverage
npm run test:coverage

# Run performance benchmarks
npm run test:performance

# Watch mode for development
npm run test:watch

Test Coverage (Phase 5 Validation Complete):

  • ✅ 72 tests passing (100% pass rate)
  • ✅ 14 basic tests - Core functionality
  • ✅ 4 health check tests - System health monitoring
  • ✅ 21 AI tools tests - AI-powered tool validation
  • ✅ 7 server integration tests - MCP protocol compliance
  • ✅ 15 edge cases tests - Error handling and edge cases
  • ✅ 11 performance tests - Performance benchmarks

Integration Testing

The TypeScript MCP server has comprehensive integration testing for real-world usage scenarios:

# From project root - run all integration tests (27/27 passing)
npm run test:integration:all

# Claude Desktop integration tests (9/9 passing)
npm run test:claude-desktop

# VS Code integration tests (11/11 passing)
npm run test:vscode

# End-to-end workflow tests (7/7 passing)
npm run test:e2e

# Quick integration validation
npm run test:quickstart

Integration Test Coverage

Claude Desktop Integration (9 tests):

  • ✅ MCP server startup and initialization
  • ✅ MCP protocol compliance (2024-11-05)
  • ✅ Tool listing and discovery (all 9 tools)
  • ✅ Search functionality with real database queries
  • ✅ Function explanation capabilities
  • ✅ Configuration file validation
  • ✅ Error handling and graceful recovery
  • ✅ Connection persistence across requests
  • ✅ Debug logging and monitoring

VS Code Integration (11 tests):

  • ✅ Workspace structure detection and analysis
  • ✅ TypeScript file parsing and understanding
  • ✅ Cross-reference finding across workspace
  • ✅ API endpoint detection and documentation
  • ✅ Code complexity analysis and metrics
  • ✅ Data flow tracing and visualization
  • ✅ Duplicate code detection and reporting
  • ✅ Refactoring suggestions and recommendations
  • ✅ Security vulnerability analysis
  • ✅ Dynamic file change handling
  • ✅ Extension configuration compatibility

End-to-End Workflows (7 tests):

  • ✅ Complete Claude Desktop session workflow
  • ✅ VS Code development workflow simulation
  • ✅ Multi-language project analysis
  • ✅ Real-time codebase change handling
  • ✅ Error recovery and service resilience
  • ✅ Performance and load testing
  • ✅ Concurrent request processing

REST API Endpoint

A new /mcp/call HTTP endpoint is available for non-MCP client access:

# Start the Fastify server
node dist/server.js

# Call MCP tools via HTTP
curl -X POST http://localhost:4000/mcp/call \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "ai_code_review",
    "arguments": {
      "code_snippet": "function example() { return 42; }",
      "review_type": "basic",
      "codebase_id": "test"
    }
  }'

Supported Tools via HTTP:

  • ai_code_review - AI-powered code review
  • bug_prediction - AI bug prediction and risk assessment
  • context_aware_code_generation - AI code generation
  • intelligent_refactoring - AI refactoring recommendations
  • technical_debt_analysis - Technical debt analysis

Error Status Codes:

  • 200 - Success
  • 400 - Invalid request (missing required fields, empty input)
  • 404 - Tool not found
  • 408 - Request timeout
  • 413 - Payload too large
  • 500 - Internal server error

Usage

CLI Commands (Fully Working)

# Index a project (supports JS/TS files)
node dist/cli/index.js index /path/to/project

# Search the indexed codebase
node dist/cli/index.js search "authentication"

# View indexing statistics
node dist/cli/index.js stats

MCP Server Integration

# Start MCP server for Claude Desktop
node dist/index.js
# Uses stdio transport by default

Claude Desktop Configuration

{
  "mcpServers": {
    "codesight": {
      "command": "node",
      "args": ["F:/path/to/project/typescript-mcp/dist/index.js"],
      "cwd": "F:/path/to/project/typescript-mcp"
    }
  }
}

AI Configuration (Phase 4.1)

Environment Variables for AI Services

# AI Provider Configuration
ANTHROPIC_API_KEY=your-anthropic-api-key     # Claude integration (recommended)
OPENAI_API_KEY=your-openai-api-key           # GPT-4 integration
OLLAMA_BASE_URL=http://localhost:11434        # Local Ollama instance

# AI Service Preferences
PREFERRED_AI_PROVIDER=anthropic-claude        # Preferred provider
ENABLE_AI_FALLBACK=true                      # Use rule-based fallback
AI_CACHE_ENABLED=true                        # Enable AI response caching
AI_TIMEOUT_MS=30000                          # AI request timeout (30s)

# AI Tool Configuration
AI_CODE_REVIEW_ENABLED=true                  # Enable AI code review
AI_REFACTORING_ENABLED=true                  # Enable AI refactoring
AI_BUG_PREDICTION_ENABLED=true               # Enable AI bug prediction
AI_CODEGEN_ENABLED=true                      # Enable AI code generation
AI_TECHNICAL_DEBT_ENABLED=true               # Enable AI technical debt analysis

AI Provider Capabilities

Provider Max Tokens Code Analysis Multimodal Latency Cost/1K Tokens
Claude 100K ✅ Excellent ❌ Medium $0.015
GPT-4 128K ✅ Very Good ✅ Medium $0.030
Ollama 8K-32K ✅ Good ❌ Fast $0.000
Rule-based 0 ✅ Basic ❌ Fast $0.000

Configuration

Configuration is managed through environment variables and src/config.ts:

// Default configuration with Phase 4.1 AI features
{
  server: {
    port: 4000,
    host: '0.0.0.0'
  },
  mcp: {
    transport: 'stdio' // or 'websocket'
  },
  rust: {
    ffiPath: '../rust-core/target/release',
    enabled: true,
    gracefulFallback: true
  },
  ai: {
    preferredProvider: 'anthropic-claude',
    enableFallback: true,
    cacheEnabled: true,
    timeout: 30000
  },
  performance: {
    useFFI: true,
    maxConcurrentFFICalls: 10,
    ffiTimeout: 5000
  }
}

Core Environment Variables

# FFI Configuration
RUST_FFI_PATH=../rust-core/target/release
ENABLE_RUST_FFI=true
FFI_GRACEFUL_FALLBACK=true
FFI_TIMEOUT=5000
MAX_CONCURRENT_FFI_CALLS=10

# Database
DATABASE_URL=sqlite://./data/codesight.db

# Performance
INDEXING_PARALLEL_WORKERS=4
INDEXING_BATCH_SIZE=500

Contract Tests

All MCP tools have comprehensive contract tests ensuring protocol compliance:

Core Tools (Phases 3.0-3.5)

  • test_search_code.ts - Natural language search validation
  • test_explain_function.ts - Function explanation validation
  • test_find_references.ts - Reference finding validation
  • test_trace_data_flow.ts - Data flow analysis validation
  • test_analyze_security.ts - Security analysis validation
  • test_get_api_endpoints.ts - API discovery validation
  • test_check_complexity.ts - Complexity analysis validation
  • test_find_duplicates.ts - Duplicate detection validation
  • test_suggest_refactoring.ts - Refactoring suggestion validation

AI Tools (Phase 4.1)

  • test_ai_code_review.ts - AI-powered code review validation
  • test_intelligent_refactoring.ts - AI refactoring validation
  • test_bug_prediction.ts - AI bug prediction validation
  • test_context_aware_codegen.ts - AI code generation validation
  • test_technical_debt_analysis.ts - Technical debt analysis validation

Run tests:

# Core tool tests
npm run test:contract

# AI tool tests
npm run test:ai-tools

# All tests
npm test

Project Structure

typescript-mcp/
├── src/
│   ├── index.ts           # MCP server entry point with AI integration
│   ├── cli/               # ✅ CLI implementation with AI commands
│   │   └── index.ts       # Working CLI commands including AI tools
│   ├── tools/             # ✅ 14 MCP tool implementations (9 core + 5 AI)
│   │   ├── Core Tools/
│   │   │   ├── search-code.ts       # Real database search
│   │   │   ├── explain-function.ts  # Function explanation
│   │   │   ├── find-references.ts   # Reference finding
│   │   │   ├── trace-data-flow.ts   # Data flow analysis
│   │   │   ├── analyze-security.ts  # Security analysis
│   │   │   ├── get-api-endpoints.ts # API discovery
│   │   │   ├── check-complexity.ts  # Complexity analysis
│   │   │   ├── find-duplicates.ts   # Duplicate detection
│   │   │   └── suggest-refactoring.ts # Refactoring suggestions
│   │   └── AI Tools (Phase 4.1)/
│   │       ├── ai-code-review.ts           # AI-powered code review
│   │       ├── intelligent-refactoring.ts  # AI refactoring analysis
│   │       ├── bug-prediction.ts           # AI bug prediction
│   │       ├── context-aware-codegen.ts    # AI code generation
│   │       └── technical-debt-analysis.ts  # AI technical debt analysis
│   ├── services/          # ✅ Core services with AI integration
│   │   ├── indexing-service.ts  # Real SQLite indexing
│   │   ├── search-service.ts    # Query processing
│   │   ├── ai-llm.ts           # Multi-provider AI service (NEW)
│   │   ├── logger.ts           # Structured logging
│   │   └── codebase-service.ts
│   ├── llm/               # 🤖 LLM provider integrations (NEW)
│   │   ├── claude.ts       # Anthropic Claude integration
│   │   ├── openai.ts       # OpenAI GPT-4 integration
│   │   ├── ollama.ts       # Ollama local models
│   │   └── router.ts       # LLM routing and fallback logic
│   ├── controllers/       # ✅ REST API controllers with AI endpoints
│   │   ├── codebase-controller.ts
│   │   ├── analysis-controller.ts
│   │   ├── search-controller.ts
│   │   ├── refactoring-controller.ts
│   │   └── ai-controller.ts    # AI tools controller (NEW)
│   ├── ffi/              # ✅ Rust FFI bridge integration
│   │   ├── index.ts      # FFI bridge interface
│   │   └── utils.ts      # FFI utilities and fallback logic
│   └── types/            # TypeScript definitions with AI types
├── tests/
│   ├── contract/         # ✅ All 14 tools tested (9 core + 5 AI)
│   │   ├── core/         # Core tool tests
│   │   └── ai/           # AI tool tests (NEW)
│   ├── integration/       # ✅ FFI bridge and AI integration tests
│   ├── performance/       # Performance benchmarks including AI workloads
│   └── ai-tools/         # AI-specific test suites (NEW)
└── dist/                 # Built JavaScript
    ├── cli/index.js      # Working CLI with AI commands
    └── index.js          # MCP server with AI integration

IndexingService Implementation

The current implementation uses a native TypeScript IndexingService with SQLite:

// Real working implementation
import { indexingService } from './services/indexing-service';

// Index a project
const fileCount = await indexingService.indexCodebase('/path/to/project');

// Search the database
const results = indexingService.searchCode('authentication', 10);

// Get statistics
const stats = indexingService.getStats();
// { total: 377, byType: { function: 175, interface: 140, ... } }

Entity Types Extracted

  • Functions: Regular functions, arrow functions, async functions
  • Classes: ES6 classes with export detection
  • Interfaces: TypeScript interfaces
  • Types: TypeScript type aliases

Real Performance Metrics

Phase 4.1 AI-Enhanced Hybrid Implementation (TypeScript + Rust FFI + AI):

Core Performance

  • Indexing Speed: 47 files in ~1-2 seconds (with Rust FFI)
  • Database Size: 377 entities in SQLite with concurrent access
  • Search Response: 20-50ms query time (with Rust FFI)
  • Memory Usage: ~25MB during indexing (optimized with Rust)
  • Startup Time: <1 second
  • Multi-Language Support: 15+ languages with Tree-sitter

AI Performance (NEW)

  • AI Code Review: 200-800ms response time (depending on provider and complexity)
  • AI Bug Prediction: 300-1200ms analysis time
  • AI Refactoring Suggestions: 250-900ms response time
  • AI Code Generation: 400-1500ms for context-aware generation
  • AI Technical Debt Analysis: 500-2000ms comprehensive analysis
  • AI Memory Overhead: ~15-30MB additional memory during AI operations

Performance Benchmarks:

Operation TypeScript Only Hybrid (TS+Rust) Hybrid + AI Improvement
File Indexing 2-3 seconds 1-2 seconds 1-2 seconds 2x faster
Search Query 50-100ms 20-50ms 20-50ms 2.5x faster
AI Code Review N/A N/A 200-800ms AI-powered insights
AI Bug Prediction N/A N/A 300-1200ms Proactive analysis
Memory Usage ~30MB ~25MB ~40-55MB Base + AI overhead
Multi-Language JS/TS only 15+ languages 15+ languages 7.5x coverage

Entity Breakdown:

  • Functions: 175 (46.4%)
  • Interfaces: 140 (37.1%)
  • Classes: 48 (12.7%)
  • Types: 14 (3.7%)

AI Performance by Provider:

Provider Response Time Quality Score Cost Offline Capability
Claude 200-600ms 9.2/10 $$$ ❌
GPT-4 300-800ms 8.8/10 $$$$ ❌
Ollama 100-400ms 7.5/10 Free ✅
Rule-based 10-50ms 6.0/10 Free ✅

Dependencies

Core Dependencies

  • @modelcontextprotocol/sdk - MCP protocol implementation
  • better-sqlite3 - SQLite database with real indexing
  • glob - File pattern matching for indexing
  • zod - Runtime type validation
  • chalk - CLI output formatting

AI/LLM Dependencies (NEW)

  • @anthropic-ai/sdk - Anthropic Claude API client
  • openai - OpenAI GPT-4 API client
  • ollama - Local Ollama integration
  • axios - HTTP client for LLM API calls
  • node-cache - AI response caching system

FFI Bridge Dependencies

  • @napi-rs/cli - Rust FFI tooling for native module compilation
  • node-gyp - Native addon build tool
  • bindings - Node.js native module binding utilities

Development Dependencies

  • typescript - TypeScript compiler
  • jest - Testing framework
  • @types/node - Node.js type definitions

Rust Workspace Dependencies (see ../rust-core/Cargo.toml)

  • napi & napi-derive - NAPI-RS for Node.js bindings
  • tree-sitter - Parser generation tool
  • rusqlite - SQLite bindings for Rust
  • serde & serde_json - Serialization

Contributing

Standard Development Workflow

  1. Ensure all tests pass: npm test
  2. Run linting: npm run lint
  3. Check types: npm run type-check
  4. Format code: npm run format
  5. Test contract compliance: npm run test:contract
  6. Test AI tools: npm run test:ai-tools
  7. Run performance benchmarks: npm run test:performance

AI/LLM Development Guidelines

  1. Configure AI providers before development:

    export ANTHROPIC_API_KEY="your-key"
export OPENAI_API_KEY="your-key"

  2. Test AI integrations:

    npm run test:ai-tools
npm run test:ai-providers

  3. Verify fallback behavior:

    ENABLE_AI_FALLBACK=false npm test

  4. AI tool development:

    • Test with all providers (Claude, GPT-4, Ollama, rule-based)
    • Ensure graceful degradation when providers are unavailable
    • Add comprehensive input validation for AI prompts
    • Include caching for expensive AI operations

FFI Bridge Development

When working on the Rust FFI bridge:

  1. Build Rust components first: cd ../rust-core && cargo build --release
  2. Test TypeScript integration: npm run test:contract
  3. Verify graceful fallback: ENABLE_RUST_FFI=false npm test
  4. Profile performance: npm run test:performance

AI Feature Guidelines

  • Context Awareness: AI tools should understand project structure and coding patterns
  • Incremental Analysis: Design for efficient incremental updates rather than full re-analysis
  • Cost Optimization: Implement caching and batching to minimize API costs
  • Quality Assurance: Validate AI suggestions with rule-based checks
  • Privacy First: Never send sensitive code to external AI services without consent

AI Ethics and Usage

This project implements AI features with the following principles:

  • Optional AI: All AI features can be disabled and work with rule-based fallbacks
  • Privacy Respecting: Code is only sent to AI providers when explicitly configured
  • Cost Transparency: AI usage costs are clearly documented and controlled
  • Quality Control: AI suggestions are validated and rated for confidence
  • User Control: Users can choose AI providers and disable features as needed

License

MIT - See LICENSE file for details

AI Provider Terms

By using AI features, you acknowledge and agree to:

  • Anthropic's Terms of Service (for Claude integration)
  • OpenAI's Terms of Service (for GPT-4 integration)
  • Applicable terms for any third-party AI providers
  • Responsible AI usage guidelines and ethical coding practices