🧠 Project Graph + Code Intelligence System

Overview

Implement continuous code indexing and project-aware intelligence to enable precise context retrieval and impact analysis.

Problem

Current Grimoire operates on text-prompt level context without understanding code structure, symbols, or relationships. This limits reasoning quality and wastes context on irrelevant information.

Solution

Build a comprehensive code intelligence system with:

Core Components

• Tree-sitter AST indexing for all supported languages
• LSP/LSIF integration where available for cross-references
• Symbol graph database (SQLite + FTS + vector index)
• Code embeddings per symbol/file/chunk for semantic search
• Continuous updates via git hooks and file watchers

CodeIndex Trait

trait CodeIndex {
    async fn index(&self, project_root: &Path) -> Result<()>;
    async fn query_symbols(&self, query: &str) -> Result<Vec<Symbol>>;
    async fn find_references(&self, symbol: &Symbol) -> Result<Vec<Reference>>;
    async fn impact_analysis(&self, diff: &Diff) -> Result<ImpactReport>;
    async fn retrieve_context(&self, files: &[PathBuf], budget: TokenBudget) -> Result<ContextBundle>;
}

Storage Schema

• Symbols table: id, name, kind, file, line, definition, signature
• References table: symbol_id, file, line, context
• Embeddings table: chunk_id, embedding, metadata
• Files table: path, hash, last_modified, ast_root

Implementation Plan

1. Start with tree-sitter for Rust, TypeScript, Python, Go
2. Add vector embeddings using sentence-transformers or similar
3. Implement continuous indexing with git hooks
4. Build retrieval system with relevance scoring
5. Add impact analysis for change suggestions

Success Criteria

• Index 10K+ file repos in <30s
• Retrieve relevant context in <100ms
• Impact analysis covers 95%+ of actual dependencies
• Context relevance score >0.8 vs manual selection

Dependencies

• tree-sitter language grammars
• sqlite-vss or external vector DB
• Language-specific analyzers
• Git integration

Timeline

6-8 weeks for MVP, 12 weeks for production-ready