Unify MCP across clients; remove legacy plugin server

2025-12-12 14:33:18 -03:00
parent 1d53ec341d
commit ec09a0a662
60 changed files with 3451 additions and 4668 deletions
--- a/docs/README.md
+++ b/docs/README.md
@@ -0,0 +1,35 @@
+# DSS Documentation (Human)
+
+This directory contains **human-facing** documentation for DSS.
+
+If you’re looking for AI/agent-oriented notes, see `docs/ai.md`.
+
+## What DSS Is
+
+DSS (Design System Server) is a toolkit that can run in two modes:
+
+1. **Local developer tool**: analyze any project on disk, extract tokens, generate Storybook stories, and expose `dss_*` tools to Claude Code via MCP.
+2. **Headless server**: run a FastAPI backend that teams (UX/QA/Admin) can access through the web Admin UI, and that AI assistants can interact with via the `/api/claude/chat` endpoint and MCP tool endpoints.
+
+## MCP across clients
+
+DSS exposes the same `dss_*` toolset to multiple AI clients via MCP:
+
+- Recommended: `./scripts/enable-mcp-clients.sh`
+- **Claude Code**: `./scripts/setup-mcp.sh` generates `.claude/mcp.json`.
+- **Codex CLI**: `codex mcp add dss -- /absolute/path/to/dss/scripts/dss-mcp`
+- **Gemini CLI**: `gemini mcp add dss /absolute/path/to/dss/scripts/dss-mcp`
+
+The Claude plugin system (commands/skills under `dss-claude-plugin/`) is Claude-specific, but DSS makes those guides/hooks available to *any* MCP client via:
+- `dss_list_guides` (use `include_meta=true` for hook metadata)
+- `dss_get_guide`
+- `dss_match_skills`
+
+## Documentation Map
+
+- `docs/quickstart.md` — install + run locally, and run the headless server.
+- `docs/architecture.md` — how CLI, API, Admin UI, and MCP fit together.
+- `docs/configuration.md` — environment variables, ports, and operational knobs.
+- `docs/storage.md` — JSON-only storage layout and `DSS_HOME` rules.
+- `docs/upgrade-notes.md` — recent upgrades and migration notes.
+- `docs/ai.md` — where the AI/agent documentation lives.
--- a/docs/ai.md
+++ b/docs/ai.md
@@ -0,0 +1,21 @@
+# AI / Agent Documentation Index
+
+This repo also includes documentation intended for AI agents (Claude Code plugin skills/commands, internal reference docs, and knowledge base files).
+
+Key entry points:
+
+- `CLAUDE.md` — instructions for Claude Code sessions and MCP setup.
+- `admin-ui/AI-REFERENCE.md` — Admin UI architecture and API usage reference.
+- `dss-claude-plugin/commands/` — Claude Code slash commands (operator runbooks).
+- `dss-claude-plugin/skills/` — skill prompts for tool-using agents.
+- `.knowledge/` — internal knowledge base documents.
+
+## Cross-model (Claude, Codex, Gemini)
+
+The DSS MCP server is client-agnostic. For non-Claude clients, the Claude plugin system (slash commands) does not load, but:
+
+- the same MCP tools are available, and
+- the plugin guides can be discovered via MCP tools:
+  - `dss_list_guides`
+  - `dss_get_guide`
+  - `dss_match_skills` (emulates skill hook matching via `globs` / `alwaysApply`)
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -0,0 +1,48 @@
+# Architecture
+
+## High-level pieces
+
+- **Python core library** (`dss/`): analysis, ingestion, Storybook generation, storage.
+- **Headless API** (`apps/api/server.py`): FastAPI server for Admin UI + AI chat + MCP endpoints.
+- **Admin UI** (`admin-ui/`): Preact app used by UX/QA/Admin (talks to the API via `/api/*`).
+- **Claude Code integration** (`dss-claude-plugin/`): Claude plugin assets (commands + skills).
+- **Local MCP stdio server** (`dss/mcp/server.py`): a minimal MCP process that exposes `dss_*` tools to Claude Code.
+
+## Execution modes
+
+### Local (developer machine)
+
+- Run `dss-cli.py` directly against any repo on disk.
+- Or run MCP locally (`python -m dss.mcp.server`) so Claude Code can call `dss_*` tools.
+
+### Headless (team server)
+
+- Run `uvicorn apps.api.server:app ...` to expose:
+  - Admin UI API (`/api/*`)
+  - AI chat (`/api/claude/chat`)
+  - MCP tool listing/execution (`/api/mcp/*`)
+- When `admin-ui/dist/` exists, the server can also serve the built Admin UI bundle as static files.
+
+### Hybrid (recommended for mixed teams)
+
+- Developers run the MCP process locally.
+- Tool execution can be proxied to a team server by setting `DSS_API_URL`.
+
+## MCP: unified tool layer
+
+All MCP-facing tool calls run through a shared registry/handler:
+
+- Registry + execution: `dss/mcp/handler.py`
+- Local MCP server (stdio): `dss/mcp/server.py`
+- Headless server endpoints: `apps/api/server.py` (`/api/mcp/*`)
+
+This avoids “two different DSS tool implementations” drifting over time.
+
+## Storage: JSON-only
+
+DSS stores state as JSON under `DSS_HOME` (see `docs/storage.md`).
+
+Why JSON:
+- portable across machines/containers
+- reviewable diffs in Git
+- simple backup/restore
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -0,0 +1,102 @@
+# Configuration
+
+## Ports (defaults)
+
+| Service | Default |
+| --- | --- |
+| API server | `6220` |
+| Admin UI (Vite dev) | `6221` |
+| MCP server (stdio process) | `6222` |
+| Storybook (DSS repo) | `6226` |
+
+Notes:
+- `./scripts/dss` runs a single-port server (UI + API) on `DSS_PORT` (defaults to `6220`).
+- Many target projects run their own Storybook on `6006`/`6007`; DSS service discovery checks those ports.
+
+## Storage
+
+- `DSS_HOME`
+  - **Server mode**: set this explicitly (e.g. `/var/lib/dss` or a project volume).
+  - **Local mode**: if a project has `./.dss/`, DSS will use it automatically; otherwise it falls back to `~/.dss`.
+
+See `docs/storage.md` for the directory layout.
+
+## AI / Integrations
+
+- `FIGMA_TOKEN` — enables live Figma extraction/sync.
+- `ANTHROPIC_API_KEY` — enables `/api/claude/chat` AI chat and tool calling.
+- `DSS_MCP_ENCRYPTION_KEY` — optional Fernet key; encrypts integration configs at rest when set.
+
+## Dev workflow (optional)
+
+- `DSS_ENABLE_DEV_COMMANDS`
+  - When set to `1`, DSS exposes dev-only MCP workflow tools that wrap local scripts (`dss_init`, `dss_reset`, `dss_services`).
+  - `scripts/dss-mcp` and `./scripts/setup-mcp.sh` set this automatically for local development.
+
+## Single-port launcher
+
+- `DSS_PORT` — port for `./scripts/dss` (single-port server wrapper).
+
+## MCP / Proxy mode
+
+- `DSS_API_URL` (or `DSS_SERVER_URL`) — if set, `dss.mcp.server` forwards tool calls to a headless DSS server.
+- `DSS_PROJECT_ID` — default project context for MCP calls (when proxying or when tools omit `path`).
+- `DSS_USER_ID` — optional user context for integrations.
+
+## MCP client setup (Claude, Codex, Gemini)
+
+All three clients can run the same DSS MCP stdio server.
+
+Quick option (configures Claude + Codex + Gemini when installed):
+
+```bash
+./scripts/enable-mcp-clients.sh
+```
+
+### Claude Code
+
+- Generate Claude Code MCP config: `./scripts/setup-mcp.sh`
+- Restart Claude Code, then run `/mcp` to verify `dss` is connected.
+
+### Codex CLI
+
+Add DSS as a global MCP server:
+
+```bash
+codex mcp add dss -- /absolute/path/to/dss/scripts/dss-mcp
+```
+
+Remove it:
+
+```bash
+codex mcp remove dss
+```
+
+### Gemini CLI
+
+Add DSS as an MCP server:
+
+```bash
+gemini mcp add dss /absolute/path/to/dss/scripts/dss-mcp
+```
+
+Remove it:
+
+```bash
+gemini mcp remove dss
+```
+
+### Notes
+
+- The Claude Code *plugin* (commands/skills under `dss-claude-plugin/`) is Claude-specific; Codex/Gemini won’t “load” it as a plugin, but they can still use the same MCP tools.
+- `scripts/dss-mcp` sets required env vars (`PYTHONPATH`, `DSS_HOME`, etc.) and runs `python -m dss.mcp.server`.
+- DSS also exposes the plugin guides/hooks to all MCP clients via:
+  - `dss_list_guides` (use `include_meta=true` to see `globs` / `alwaysApply` / `arguments`)
+  - `dss_get_guide`
+  - `dss_match_skills`
+
+## Scripts
+
+- `./scripts/setup-mcp.sh`:
+  - generates `.claude/mcp.json` to run the local MCP stdio server (`python -m dss.mcp.server`)
+  - `--api-url <url>` — enable proxy mode for the local MCP server
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -0,0 +1,108 @@
+# Quickstart
+
+## Prerequisites
+
+- Python 3.10+
+- Node.js 18+ (for `admin-ui` and Storybook tooling)
+
+## 1) Install Python deps
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+```
+
+## 2) Local developer workflow (CLI only)
+
+Run analysis for any project on disk:
+
+```bash
+./dss-cli.py analyze --project-path /absolute/path/to/your-project
+```
+
+This writes a portable JSON artifact:
+- `<project>/.dss/analysis_graph.json`
+
+Generate Storybook stories for a project:
+
+```bash
+./dss-cli.py setup-storybook --action generate --project-path /absolute/path/to/your-project
+```
+
+## 3) Local developer workflow (Claude Code via MCP)
+
+Generate Claude Code MCP config:
+
+```bash
+./scripts/setup-mcp.sh
+```
+
+Then restart Claude Code and run `/mcp` to confirm the `dss` server is connected.
+
+### Enable MCP for all supported clients (recommended)
+
+```bash
+./scripts/enable-mcp-clients.sh
+```
+
+### Proxy mode (MCP process local, tools executed on a server)
+
+```bash
+./scripts/setup-mcp.sh --api-url https://your-dss-server.example.com
+```
+
+Set `DSS_PROJECT_ID` in the MCP server environment when you want tools to default to a specific registered project.
+
+## 4) Headless server (API for Admin UI + teams)
+
+Choose one of the following:
+
+### Option A: Dev mode (Vite Admin UI + API)
+
+Start the API server:
+
+```bash
+source .venv/bin/activate
+PYTHONPATH="$PWD:$PWD/apps/api" uvicorn apps.api.server:app --host 0.0.0.0 --port 6220
+```
+
+Start the Admin UI (dev mode with `/api` proxy to `:6220`):
+
+```bash
+cd admin-ui
+npm install
+npm run dev
+```
+
+Admin UI: `http://localhost:6221`  
+API: `http://localhost:6220`
+
+### Option B: Single-port server (serves built Admin UI)
+
+Build the Admin UI once:
+
+```bash
+cd admin-ui
+npm install
+npm run build
+cd ..
+```
+
+Start DSS (serves UI + API from one process):
+
+```bash
+./scripts/dss start
+```
+
+Dashboard: `http://localhost:6220`  
+API: `http://localhost:6220/api`
+
+## 5) Common environment variables
+
+- `DSS_HOME` — where DSS stores JSON data (defaults to `./.dss` when present, else `~/.dss`)
+- `FIGMA_TOKEN` — enables live Figma tooling
+- `ANTHROPIC_API_KEY` — enables AI chat/tool calling on the headless server
+- `DSS_API_URL` — enables MCP proxy mode (local MCP → headless server)
+
+See `docs/configuration.md` for the full list and details.
--- a/docs/storage.md
+++ b/docs/storage.md
@@ -0,0 +1,59 @@
+# Storage (JSON-only)
+
+## `DSS_HOME` resolution
+
+DSS chooses a storage root in this order:
+
+1. `$DSS_HOME` if set
+2. `./.dss` if it exists in the current working directory
+3. `~/.dss`
+
+This supports:
+- **project-local** storage (recommended when running DSS inside a repo)
+- **shared user** storage (default fallback)
+- **server** storage (explicit `DSS_HOME` volume)
+
+## Directory layout
+
+Most persistent JSON state lives under:
+
+`$DSS_HOME/data/`
+
+```
+data/
+  _system/
+    activity/            # JSONL activity logs (by day)
+    cache/               # TTL cache entries (JSON)
+    users/               # auth users (JSON-only)
+    integration_health.json
+  projects/
+    <project_id>/
+      manifest.json
+      components/
+      figma/
+      integrations.json
+      metrics/
+      styles/
+      tokens/
+  teams/
+```
+
+## Concurrency / locking
+
+The JSON store uses simple file locking (`*.lock`) to reduce corruption when multiple processes write concurrently.
+
+## Project-local artifacts
+
+In addition to `$DSS_HOME/data/`, DSS also writes project-local artifacts under a repo’s `.dss/` directory, for example:
+
+- `.dss/analysis_graph.json` — portable analysis output (commit-friendly).
+- `.dss/config.json` — per-project DSS configuration (used by `ConfigService`).
+- `.dss/runtime-config.json` — runtime/server preferences (when using the headless server wrappers).
+
+## What is safe to commit
+
+Typical “commit-worthy” artifacts:
+- `<project>/.dss/analysis_graph.json` (generated analysis output)
+
+Typical “do not commit” data:
+- per-user caches and secrets under `$DSS_HOME/data/_system/cache`
--- a/docs/upgrade-notes.md
+++ b/docs/upgrade-notes.md
@@ -0,0 +1,61 @@
+# Upgrade Notes
+
+This file summarizes major upgrades applied to DSS to support:
+- JSON-only, portable storage
+- a single MCP tool layer used by local + server + plugin
+- local developer usage across many projects **and** a headless server for UX/QA/Admin
+
+## Key upgrades
+
+### 1) Unified MCP tool layer (`dss.mcp`)
+
+Created a shared tool registry/execution layer used by:
+- Headless server (`apps/api/server.py` → `/api/mcp/*`)
+- Local MCP stdio server (`python -m dss.mcp.server`)
+
+Proxy mode is supported via `DSS_API_URL`.
+
+### 2) JSON-only storage (removed SQLite)
+
+All state is stored as JSON under `DSS_HOME`:
+- Project data: `$DSS_HOME/data/projects/<project_id>/...`
+- System data: `$DSS_HOME/data/_system/...`
+
+Auth user storage is JSON-based (`dss/storage/json_store.py:Users`).
+
+### 3) Consistent storage root (`DSS_HOME`)
+
+Storage now resolves in this order:
+1. `DSS_HOME`
+2. `./.dss` (project-local, if present)
+3. `~/.dss`
+
+### 4) Storybook generation reliability
+
+Story generation now:
+- scans component directories recursively
+- writes the correct story extension (`.stories.jsx` vs `.stories.tsx`)
+- avoids deleting user-authored stories by only clearing files with a DSS marker
+
+### 5) Headless server serves built Admin UI
+
+When `admin-ui/dist/index.html` exists, the headless server now serves the built Admin UI bundle as static files (so teams can access the web dashboard from the server port).
+
+### 6) CLI analysis now produces portable JSON output
+
+`./dss-cli.py analyze --project-path <path>` writes:
+- `<project>/.dss/analysis_graph.json`
+
+This is intended to be commit-friendly and shareable across machines.
+
+### 7) Removed deprecated `dss-setup`
+
+The deprecated Claude Code command `/dss-setup` has been removed. Use `/dss-init`.
+
+## Migration notes
+
+- If you previously relied on `.dss/dss.db` or `DSS_DATABASE`: it is no longer used.
+- If you previously imported `dss_mcp` or used `dss/mcp_server/*`: use `dss/mcp/*`.
+- If you previously referenced `dss-claude-plugin/servers/*` for MCP: use `python -m dss.mcp.server`.
+- Re-run `./scripts/setup-mcp.sh` after pulling to refresh `.claude/mcp.json`.
+- If you used the single-port launcher on `:3456`, note `./scripts/dss` now defaults to `:6220` (override with `DSS_PORT` or update your reverse proxy).