# Documentation Synchronization System

**Status**: ✅ Phase 1 Complete
**Version**: 1.0.0
**Last Updated**: 2025-12-07

## Overview

Automated documentation synchronization system that keeps code and documentation in sync using git hooks and code extractors.

## Problem Solved

- **Documentation drift**: Manual docs get stale as code changes
- **Multiple sources**: .knowledge/ JSON, MCP memory, markdown files
- **Temporary artifacts**: .dss/ accumulates session files
- **Manual updates**: Time-consuming and error-prone

## Solution

5-component automated system:

1. **Documentation Manifest** - Maps code files to documentation targets
2. **Git Hooks** - Triggers doc generation on commits
3. **Documentation Generators** - Extract structured data from code
4. **MCP Memory Sync** - Keep knowledge graph synchronized (Phase 3)
5. **Cleanup Automation** - Archive old session files (Phase 4)

## Quick Start

### Installation

```bash
# Install git hooks
cd /home/overbits/dss
.dss/doc-sync/install_hooks.sh
```

### Manual Sync

```bash
# Run all generators manually
python3 .dss/doc-sync/doc_sync_runner.py run --trigger manual

# Validate manifest
python3 .dss/doc-sync/doc_sync_runner.py validate
```

### Git Integration

After installation, git hooks automatically:

- **pre-commit**: Validates manifest, warns about doc changes
- **post-commit**: Regenerates documentation from code

## Architecture

```
.dss/doc-sync/
├── manifest.json              # Central configuration
├── doc_sync_runner.py         # Main orchestrator
├── generators/                # Code extractors
│   ├── __init__.py
│   ├── base_generator.py     # Abstract base class
│   ├── api_extractor.py      # FastAPI endpoints → JSON
│   └── mcp_extractor.py      # MCP tools → JSON
├── validators/                # Schema validators (Phase 2)
├── sync/                      # MCP memory sync (Phase 3)
└── hooks/                     # Git hook templates
    ├── pre-commit
    ├── post-commit
    └── install_hooks.sh
```

## Configuration

Edit `.dss/doc-sync/manifest.json` to configure:

- **code_mappings**: Map source files to documentation targets
- **generators**: Enable/disable specific extractors
- **triggers**: When to run generators (post-commit, manual)
- **cleanup_policy**: Archival rules for .dss/ artifacts

### Example Code Mapping

```json
{
  "pattern": "tools/api/server.py",
  "extracts_to": ".knowledge/dss-architecture.json",
  "generator": "api_extractor",
  "mcp_entities": ["DSS_FastAPI_Server", "DSS_API_Endpoints"],
  "triggers": ["post-commit", "manual"]
}
```

## Generators

### API Extractor

**Purpose**: Extract FastAPI route definitions

**Extracts**:
- Route paths and HTTP methods
- Function names and docstrings
- Route parameters
- Static file mounts

**Output**: `.knowledge/dss-architecture.json` (modules list)

### MCP Extractor

**Purpose**: Extract MCP tool definitions

**Extracts**:
- Tool names and descriptions
- Input parameters
- Tool categories
- Handler locations

**Output**: `.knowledge/mcp-tools.json`

## Git Hooks

### pre-commit

**Actions**:
- Validates manifest.json syntax
- Warns if .knowledge/ changes without code changes
- Blocks commit on validation errors (optional)

### post-commit

**Actions**:
- Runs configured generators
- Updates .knowledge/ files
- Reports changes for manual commit

**Note**: Generated .knowledge/ changes are NOT auto-committed. You must:

```bash
git add .knowledge/
git commit -m "docs: update knowledge base (auto-generated)"
```

## Backup Strategy

All .knowledge/ files are automatically backed up before updates:

```
.dss/backups/knowledge/
├── dss-architecture_20251207_182749.json
├── mcp-tools_20251207_182749.json
└── ...
```

Backups are timestamped and retained for recovery.

## Testing

```bash
# Validate manifest
python3 .dss/doc-sync/doc_sync_runner.py validate

# Test generators (dry run)
python3 .dss/doc-sync/doc_sync_runner.py run --trigger manual

# Check generated files
cat .knowledge/dss-architecture.json
cat .knowledge/mcp-tools.json
```

## Implementation Status

### Phase 1: Foundation ✅ COMPLETE

- [x] Directory structure
- [x] manifest.json configuration
- [x] Base DocGenerator class
- [x] API extractor
- [x] MCP extractor
- [x] Git hooks (pre-commit, post-commit)
- [x] Installation script
- [x] Testing and validation

### Phase 2: Additional Generators (Planned)

- [ ] Component extractor (UI components)
- [ ] Architecture analyzer (module structure)
- [ ] Dependency graph generator

### Phase 3: MCP Memory Sync (Planned)

- [ ] MCP sync engine
- [ ] Incremental updates
- [ ] Provenance tracking
- [ ] Git hook integration

### Phase 4: Cleanup Automation (Planned)

- [ ] Artifact archiver
- [ ] Cron job setup
- [ ] Compression support
- [ ] Retention policies

## Success Metrics

✅ **Achieved** (Phase 1):
- Code changes automatically trigger doc generation
- .knowledge/ files updated from source code
- Backups created before updates
- Git hooks installed and working
- <2 second overhead on git commit

**Target** (Future phases):
- MCP memory graph synchronized
- .dss/ artifacts auto-archived monthly
- No manual doc updates required

## Troubleshooting

### Generators fail

```bash
# Check Python path
python3 -c "import sys; print(sys.path)"

# Run with debug logging
python3 -v .dss/doc-sync/doc_sync_runner.py run
```

### Git hooks not running

```bash
# Check hook installation
ls -la .git/hooks/
cat .git/hooks/post-commit

# Reinstall hooks
.dss/doc-sync/install_hooks.sh
```

### .knowledge/ files not updating

```bash
# Check manifest configuration
python3 .dss/doc-sync/doc_sync_runner.py validate

# Check file permissions
ls -la .knowledge/

# Manual sync
python3 .dss/doc-sync/doc_sync_runner.py run --trigger manual
```

## Uninstallation

```bash
# Remove git hooks
rm .git/hooks/pre-commit
rm .git/hooks/post-commit

# Restore backups if needed
mv .git/hooks/pre-commit.backup .git/hooks/pre-commit
mv .git/hooks/post-commit.backup .git/hooks/post-commit
```

## Future Enhancements

- **Watch mode**: Continuous doc generation during development
- **Diff reports**: Show what changed in documentation
- **Conflict resolution**: Handle merge conflicts in generated docs
- **CI/CD integration**: Automated doc validation in pipelines
- **Web dashboard**: Visualize documentation health

## Links

- [Manifest Configuration](manifest.json)
- [Base Generator](generators/base_generator.py)
- [API Extractor](generators/api_extractor.py)
- [MCP Extractor](generators/mcp_extractor.py)
- [Installation Script](install_hooks.sh)

---

**Generated**: 2025-12-07
**Documentation Sync**: Phase 1 Complete