CmdForge/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

CmdForge is a lightweight personal tool builder for AI-powered CLI commands. It lets users create custom terminal commands that call AI providers, chain prompts with Python code steps, and use them like any Unix pipe command.

## Development Commands

```bash
# Install for development
pip install -e ".[dev]"

# Run all unit tests (excluding integration tests that need a server)
pytest tests/ -m "not integration"

# Run a specific test file
pytest tests/test_runner.py -v

# Run a specific test class or method
pytest tests/test_runner.py::TestSubstituteVariables -v
pytest tests/test_runner.py::TestSubstituteVariables::test_simple_substitution -v

# Run with coverage
pytest tests/ --cov=cmdforge --cov-report=html

# Run integration tests (requires local registry server at localhost:5000)
python -m cmdforge.registry.app  # Start server first
pytest tests/test_registry_integration.py -v -m integration

# CLI entry points
cmdforge              # Main CLI / GUI launcher
cf                    # Interactive tool picker (fzf-style)
python -m cmdforge.cli  # Alternative CLI invocation
```

## Architecture

### Core Modules (`src/cmdforge/`)

- **cli/**: CLI commands entry points (`cmdforge` command). Routes subcommands: list, create, edit, delete, test, run, ui, docs, check, refresh, providers, registry, collections, deps, install, lock, verify, add, remove, init, config, settings, system-deps
- **tool.py**: Tool definition dataclasses (`Tool`, `ToolArgument`, `PromptStep`, `CodeStep`, `ToolStep`, `McpStep`), YAML config loading/saving, wrapper script generation
- **runner.py**: Execution engine. Runs tool steps sequentially, handles variable substitution (`{input}`, `{varname}`), executes Python code steps via `exec()`, handles nested tool calls with depth limit (MAX_TOOL_DEPTH=10)
- **resolver.py**: Tool resolution. `resolve_tool()` searches: project manifest → local tools → owner/name → registry. Returns `ResolvedTool` with path info
- **collection.py**: Collection management (`Collection` dataclass, `resolve_tool_references()`, `classify_tool_reference()`), local collection storage in `~/.cmdforge/collections/`
- **providers.py**: Provider abstraction. Supports subprocess CLI tools, OpenAI-compatible HTTP APIs, experimental PTY wrappers, fallback chains, and tool/MCP-server allowlists. Auto-discovers installed providers on first run. Config in `~/.cmdforge/providers.yaml` with versioned migration.
- **skills.py**: Validated Agent Skills loader for per-provider `SKILL.md` context under `~/.cmdforge/providers/<name>/skills/`
- **mcp_client.py / mcp_server.py**: Stdio MCP client/server integration, configuration, schema discovery, and exposure policy
- **profiles.py**: AI persona profiles with system prompts, stored in `~/.cmdforge/profiles/`
- **manifest.py**: Project manifest (`cmdforge.yaml`) for declaring tool dependencies with version constraints
- **lockfile.py**: Lock file support for reproducible installs (`cmdforge.lock`)
- **registry_client.py**: Client for registry API (search, publish, download, authentication)
- **config.py**: Global configuration management (registry URL, auth tokens)
- **system_deps.py**: System package dependency management (apt/dnf/pacman detection and installation)
- **dependency_graph.py**: Dependency graph resolution for meta-tools
- **hash_utils.py**: Hash verification utilities for tool integrity
- **gui/**: PySide6 desktop GUI
  - **main_window.py**: Main application window with sidebar navigation
  - **pages/**: Welcome page, Tools page, Tool Builder, Registry browser, Collections page, Providers management, Profiles
  - **dialogs/**: Step editors (prompt/code/tool), Argument editor, Provider dialog, Provider Install dialog, Connect/Publish dialogs, Test Step dialog, Settings dialog, System Dep dialog, Help dialogs, Review/Issue dialogs
  - **widgets/**: Flow graph visualization (`flow_graph.py`), Icon utilities (`icons.py`)

### Key Paths

- **Tools storage**: `~/.cmdforge/<toolname>/config.yaml`
- **Tool defaults**: `~/.cmdforge/<toolname>/defaults.yaml` (optional, published with tool)
- **Tool settings**: `~/.cmdforge/<toolname>/settings.yaml` (user overrides, auto-created from defaults)
- **Wrapper scripts**: `~/.local/bin/<toolname>` (auto-generated bash scripts)
- **Provider config**: `~/.cmdforge/providers.yaml`
- **Collections storage**: `~/.cmdforge/collections/<name>.yaml`

### Tool Structure

Tools are YAML configs with:
- `name`, `description`, `category`
- `arguments`: Custom flags with defaults (e.g., `--max` → `{max}`)
- `steps`: Ordered list of `prompt`, `code`, or `tool` steps
- `output`: Template for final output (e.g., `"{response}"`)
- `input_schema`, `output_schema`: Optional tool-level JSON Schema contracts;
  schemas are validated when tools are loaded or created
- `deprecated`, `deprecated_message`, `replacement`: Validated migration metadata;
  replacements may be local names or `owner/tool` registry references

Run `cmdforge inspect <tool> [--registry]` for the shared local preflight
report. `cmdforge registry publish <path> --dry-run` runs local checks first
and, when authenticated, the registry's publish-time checks without publishing.
Normal publication uses the same checks as Stage 1, asks for confirmation in an
interactive terminal, and then performs the Stage 2 mutation. The registry
reruns deterministic conformance authoritatively and stores immutable audit and
quality evidence for the published version.
Inspect also shows inferred contract proposals and runs deterministic contract
conformance when both tool schemas are explicit. Prompt outputs are synthesized
from their schemas; code, nested-tool, and MCP steps are reported as unsupported
instead of being executed implicitly.
Use `cmdforge inspect <tool> --save-baseline` to explicitly promote passing
evidence to `conformance.json`. Later inspections and local publish dry-runs
rerun the stored synthetic inputs and report state, coverage, contract, and
normalized output changes. ToolStep preflight also compares the values the
runner supplies with the called tool's input contract.
Quality headlines exclude categories with missing evidence and are accompanied
by evidence coverage plus checked/not-tested category states.

### Step Types

1. **Prompt Step**: Calls AI provider with template, stores result in `output_var`
   - `profile`: AI persona (system prompt prefix)
   - `strip_fences`: Remove markdown code fences from output
   - `output_schema`: JSON schema for validated structured responses
   - `max_tokens`: Max output tokens (provider-dependent, e.g., 4096 for haiku)
   - `plain_text`: Bypass structured output enforcement
2. **Code Step**: Executes Python code via `exec()`, captures specified variables (comma-separated for multiple outputs)
3. **Tool Step**: Calls another tool (meta-tools), supports `input_template`, `args`, and recursive agent context (`provider`, `profile`, `skills`, `tools`). Provider and delegated allowlists are both enforced before resolution.
4. **MCP Step**: Calls a tool on a configured MCP server. Provider `mcp_servers` allowlists are enforced before connecting.

### Variable Flow

Variables are passed between steps:
- `{input}` - always available (stdin/file content)
- `{argname}` - from tool arguments
- `{step_output_var}` - from previous step's `output_var`
- `{settings.key}` - from tool's settings.yaml (top-level scalars only in templates)
- `settings['key']` - full dict access in code steps

Variable substitution is handled in `runner.py:substitute_variables()`. Settings are loaded from `settings.yaml` if it exists, otherwise an empty dict is used.

## Provider System

Providers wrap AI CLIs or compatible HTTP APIs. Defined in `~/.cmdforge/providers.yaml`:

```yaml
version: 2
providers:
  - name: claude
    command: "claude -p"
    description: "Anthropic Claude"
    type: subprocess
  - name: openrouter
    command: "https://openrouter.ai/api/v1"
    description: "OpenRouter API"
    type: api
    model: openrouter/auto-beta
    api_key_env: OPENROUTER_API_KEY
  - name: mock
    command: "mock"
    description: "Mock provider for testing"
```

Provider fields:
- `name`: Provider identifier used in tool configs
- `command`: Shell command (subprocess/pty) or endpoint URL (api)
- `type`: `subprocess` (default), `api` (OpenAI-compatible HTTP), or `pty` (interactive CLI, experimental)
- `model`: Model ID for api-type providers
- `api_key_env`: Environment variable holding the API key for api-type providers
- `description`: Optional human-readable description
- `fallback`: Optional provider to try if this one fails
- `fallback_chain`: Ordered list of providers for multi-step fallback
- `tags`: List of strings (e.g. `["free", "code", "local"]`)
- `pty_config`: Dict with `prompt_pattern`, `response_pattern`, `exit_command` for pty providers
- `install`: Optional structured install metadata dict
- `tools`: Optional CmdForge tool allowlist; `null` allows all and `[]` denies all
- `mcp_servers`: Optional MCP server allowlist; `null` allows all and `[]` denies all

The `mock` provider is built-in for testing without API calls. Use `--provider mock` or `--dry-run` flags when testing tools.

Provider CLI commands:
- `cmdforge providers list` - List all providers and their status
- `cmdforge providers check` - Check which providers are available
- `cmdforge providers add <name> <command>` - Add/update a provider (supports `--type`, `--model`, `--api-key-env`, `--tag`, `--fallback`, `--fallback-chain`)
- `cmdforge providers remove <name>` - Remove a provider
- `cmdforge providers test <name>` - Test a provider
- `cmdforge providers discover [--add]` - Scan system for installed CLIs and API keys
- `cmdforge providers install` - Interactive guide to install AI providers
- `cmdforge providers for-tools <tool> [tools...]` - List providers used by tools (with `--warm` to pre-load local models)

## Web UI & Registry

CmdForge includes a web interface and tool registry:

### Web Modules (`src/cmdforge/web/`)

- **app.py**: Flask app factory, registers blueprints
- **routes.py**: Main web routes (docs, tutorials, tools, etc.)
- **auth.py**: Authentication middleware and decorators
- **sessions.py**: Session management
- **email.py**: Email utilities (password reset, notifications)
- **seo.py**: SEO utilities (sitemap, meta tags)
- **filters.py**: Jinja2 filters (timeago, markdown, etc.)
- **docs_content.py**: Documentation and tutorial content
- **forum/**: Community forum blueprint
  - **models.py**: Forum database schema (categories, topics, replies)
  - **routes.py**: Forum routes (/forum, /forum/c/<slug>, /forum/t/<id>)

### Registry Modules (`src/cmdforge/registry/`)

- **app.py**: Flask-based Registry API (tool publishing, search, downloads, authentication, rate limiting)
- **db.py**: SQLite schema and queries (`connect_db()`, `query_one()`, `query_all()`)
- **embeddings.py**: Semantic search with vector embeddings for tool discovery
- **sync.py**: Git-based tool sync from Gitea repository
- **categorize.py**: Auto-categorization of tools based on content
- **rate_limit.py**: Rate limiting for API endpoints
- **scrutiny.py**: Tool vetting (honesty, transparency, scope, efficiency checks)
- **similarity.py**: Duplicate detection via embedding similarity
- **stats.py**: Registry statistics and metrics
- **settings.py**: Server-side settings management

### Key URLs

- `/forum` - Community forum
- `/docs` - Documentation
- `/tutorials` - Tutorial guides
- `/tools` - Tool registry browser

### Running the Web UI

```bash
# Development
python -m cmdforge.web.app

# Production (example)
CMDFORGE_REGISTRY_DB=/path/to/db PORT=5050 python -m cmdforge.web.app
```

## Testing Conventions

Tests use `pytest` without a shared conftest.py. Common patterns:

**Mocking strategy:**
- **File system**: Use `tmp_path` fixture and patch module-level paths (`TOOLS_DIR`, `BIN_DIR`, `PROVIDERS_FILE`)
- **Subprocess calls**: Mock `subprocess.run` and `shutil.which`
- **Provider calls**: Mock `call_provider` to return `ProviderResult(text="...", success=True)`
- **Registry calls**: Mock `requests` for API tests

**Example fixture pattern:**
```python
@pytest.fixture
def temp_providers_file(tmp_path):
    providers_file = tmp_path / "providers.yaml"
    with patch('cmdforge.providers.PROVIDERS_FILE', providers_file):
        yield providers_file
```

Integration tests are marked with `@pytest.mark.integration` and require a running registry server.

## Infrastructure Documentation

For deployment, server details, and operations, see the `docs/` folder:

- **docs/servers.md** - Server IPs (192.168.0.162), SSH access, paths, service commands
- **docs/deployment.md** - Architecture diagram, deploy process, systemd service config
- **docs/maintenance.md** - Backups, updates, troubleshooting
- **docs/architecture.md** - Module structure, data flow diagrams

### Production Server Quick Reference

| Property | Value |
|----------|-------|
| Server | 192.168.0.162 (OpenMediaVault) |
| SSH | `ssh rob@192.168.0.162` |
| App Path | `/srv/mergerfs/data_pool/home/rob/cmdforge-registry/` |
| Service | `systemctl --user status cmdforge-web` |
| Public URL | https://cmdforge.brrd.tech |
| Port | 5050 (via Cloudflare tunnel) |