dss/.dss/doc-sync/README.md

Documentation Synchronization System

Status: ✅ Phase 1 Complete Version: 1.0.0 Last Updated: 2025-12-07

Overview

Automated documentation synchronization system that keeps code and documentation in sync using git hooks and code extractors.

Problem Solved

• Documentation drift: Manual docs get stale as code changes
• Multiple sources: .knowledge/ JSON, MCP memory, markdown files
• Temporary artifacts: .dss/ accumulates session files
• Manual updates: Time-consuming and error-prone

Solution

5-component automated system:

1. Documentation Manifest - Maps code files to documentation targets
2. Git Hooks - Triggers doc generation on commits
3. Documentation Generators - Extract structured data from code
4. MCP Memory Sync - Keep knowledge graph synchronized (Phase 3)
5. Cleanup Automation - Archive old session files (Phase 4)

Quick Start

Installation

# Install git hooks
cd /home/overbits/dss
.dss/doc-sync/install_hooks.sh

Manual Sync

# Run all generators manually
python3 .dss/doc-sync/doc_sync_runner.py run --trigger manual

# Validate manifest
python3 .dss/doc-sync/doc_sync_runner.py validate

Git Integration

After installation, git hooks automatically:

• pre-commit: Validates manifest, warns about doc changes
• post-commit: Regenerates documentation from code

Architecture

.dss/doc-sync/
├── manifest.json              # Central configuration
├── doc_sync_runner.py         # Main orchestrator
├── generators/                # Code extractors
│   ├── __init__.py
│   ├── base_generator.py     # Abstract base class
│   ├── api_extractor.py      # FastAPI endpoints → JSON
│   └── mcp_extractor.py      # MCP tools → JSON
├── validators/                # Schema validators (Phase 2)
├── sync/                      # MCP memory sync (Phase 3)
└── hooks/                     # Git hook templates
    ├── pre-commit
    ├── post-commit
    └── install_hooks.sh

Configuration

Edit .dss/doc-sync/manifest.json to configure:

• code_mappings: Map source files to documentation targets
• generators: Enable/disable specific extractors
• triggers: When to run generators (post-commit, manual)
• cleanup_policy: Archival rules for .dss/ artifacts

Example Code Mapping

{
  "pattern": "tools/api/server.py",
  "extracts_to": ".knowledge/dss-architecture.json",
  "generator": "api_extractor",
  "mcp_entities": ["DSS_FastAPI_Server", "DSS_API_Endpoints"],
  "triggers": ["post-commit", "manual"]
}

Generators

API Extractor

Purpose: Extract FastAPI route definitions

Extracts:

• Route paths and HTTP methods
• Function names and docstrings
• Route parameters
• Static file mounts

Output: .knowledge/dss-architecture.json (modules list)

MCP Extractor

Purpose: Extract MCP tool definitions

Extracts:

• Tool names and descriptions
• Input parameters
• Tool categories
• Handler locations

Output: .knowledge/mcp-tools.json

Git Hooks

pre-commit

Actions:

• Validates manifest.json syntax
• Warns if .knowledge/ changes without code changes
• Blocks commit on validation errors (optional)

post-commit

Actions:

• Runs configured generators
• Updates .knowledge/ files
• Reports changes for manual commit

Note: Generated .knowledge/ changes are NOT auto-committed. You must:

git add .knowledge/
git commit -m "docs: update knowledge base (auto-generated)"

Backup Strategy

All .knowledge/ files are automatically backed up before updates:

.dss/backups/knowledge/
├── dss-architecture_20251207_182749.json
├── mcp-tools_20251207_182749.json
└── ...

Backups are timestamped and retained for recovery.

Testing

# Validate manifest
python3 .dss/doc-sync/doc_sync_runner.py validate

# Test generators (dry run)
python3 .dss/doc-sync/doc_sync_runner.py run --trigger manual

# Check generated files
cat .knowledge/dss-architecture.json
cat .knowledge/mcp-tools.json

Implementation Status

Phase 1: Foundation ✅ COMPLETE

• Directory structure
• manifest.json configuration
• Base DocGenerator class
• API extractor
• MCP extractor
• Git hooks (pre-commit, post-commit)
• Installation script
• Testing and validation

Phase 2: Additional Generators (Planned)

• Component extractor (UI components)
• Architecture analyzer (module structure)
• Dependency graph generator

Phase 3: MCP Memory Sync (Planned)

• MCP sync engine
• Incremental updates
• Provenance tracking
• Git hook integration

Phase 4: Cleanup Automation (Planned)

• Artifact archiver
• Cron job setup
• Compression support
• Retention policies

Success Metrics

✅ Achieved (Phase 1):

• Code changes automatically trigger doc generation
• .knowledge/ files updated from source code
• Backups created before updates
• Git hooks installed and working
• <2 second overhead on git commit

Target (Future phases):

• MCP memory graph synchronized
• .dss/ artifacts auto-archived monthly
• No manual doc updates required

Troubleshooting

Generators fail

# Check Python path
python3 -c "import sys; print(sys.path)"

# Run with debug logging
python3 -v .dss/doc-sync/doc_sync_runner.py run

Git hooks not running

# Check hook installation
ls -la .git/hooks/
cat .git/hooks/post-commit

# Reinstall hooks
.dss/doc-sync/install_hooks.sh

.knowledge/ files not updating

# Check manifest configuration
python3 .dss/doc-sync/doc_sync_runner.py validate

# Check file permissions
ls -la .knowledge/

# Manual sync
python3 .dss/doc-sync/doc_sync_runner.py run --trigger manual

Uninstallation

# Remove git hooks
rm .git/hooks/pre-commit
rm .git/hooks/post-commit

# Restore backups if needed
mv .git/hooks/pre-commit.backup .git/hooks/pre-commit
mv .git/hooks/post-commit.backup .git/hooks/post-commit

Future Enhancements

• Watch mode: Continuous doc generation during development
• Diff reports: Show what changed in documentation
• Conflict resolution: Handle merge conflicts in generated docs
• CI/CD integration: Automated doc validation in pipelines
• Web dashboard: Visualize documentation health

Links

• Manifest Configuration
• Base Generator
• API Extractor
• MCP Extractor
• Installation Script

---

Generated: 2025-12-07 Documentation Sync: Phase 1 Complete