CmdForge/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

CmdForge is a lightweight personal tool builder for AI-powered CLI commands. It lets users create custom terminal commands that call AI providers, chain prompts with Python code steps, and use them like any Unix pipe command.

## Development Commands

```bash
# Install for development
pip install -e ".[dev]"

# Run all unit tests (excluding integration tests that need a server)
pytest tests/ -m "not integration"

# Run a specific test file
pytest tests/test_runner.py -v

# Run a specific test class or method
pytest tests/test_runner.py::TestSubstituteVariables -v
pytest tests/test_runner.py::TestSubstituteVariables::test_simple_substitution -v

# Run with coverage
pytest tests/ --cov=cmdforge --cov-report=html

# Run integration tests (requires local registry server at localhost:5000)
python -m cmdforge.registry.app  # Start server first
pytest tests/test_registry_integration.py -v -m integration

# CLI entry points
cmdforge              # Main CLI / GUI launcher
cf                    # Interactive tool picker (fzf-style)
python -m cmdforge.cli  # Alternative CLI invocation
```

## Architecture

### Core Modules (`src/cmdforge/`)

- **cli/**: CLI commands entry points (`cmdforge` command). Routes subcommands: list, create, edit, delete, test, run, ui, docs, check, refresh, providers, registry, collections, deps, install, lock, verify, add, remove, init, config, settings, system-deps
- **tool.py**: Tool definition dataclasses (`Tool`, `ToolArgument`, `PromptStep`, `CodeStep`, `ToolStep`), YAML config loading/saving, wrapper script generation
- **runner.py**: Execution engine. Runs tool steps sequentially, handles variable substitution (`{input}`, `{varname}`), executes Python code steps via `exec()`, handles nested tool calls with depth limit (MAX_TOOL_DEPTH=10)
- **resolver.py**: Tool resolution. `resolve_tool()` searches: project manifest → local tools → owner/name → registry. Returns `ResolvedTool` with path info
- **collection.py**: Collection management (`Collection` dataclass, `resolve_tool_references()`, `classify_tool_reference()`), local collection storage in `~/.cmdforge/collections/`
- **providers.py**: Provider abstraction. Calls AI CLI tools via subprocess, reads provider configs from `~/.cmdforge/providers.yaml`
- **profiles.py**: AI persona profiles with system prompts, stored in `~/.cmdforge/profiles/`
- **manifest.py**: Project manifest (`cmdforge.yaml`) for declaring tool dependencies with version constraints
- **lockfile.py**: Lock file support for reproducible installs (`cmdforge.lock`)
- **registry_client.py**: Client for registry API (search, publish, download, authentication)
- **config.py**: Global configuration management (registry URL, auth tokens)
- **system_deps.py**: System package dependency management (apt/dnf/pacman detection and installation)
- **dependency_graph.py**: Dependency graph resolution for meta-tools
- **hash_utils.py**: Hash verification utilities for tool integrity
- **gui/**: PySide6 desktop GUI
  - **main_window.py**: Main application window with sidebar navigation
  - **pages/**: Welcome page, Tools page, Tool Builder, Registry browser, Collections page, Providers management, Profiles
  - **dialogs/**: Step editors (prompt/code/tool), Argument editor, Provider dialog, Connect/Publish dialogs, Test Step dialog, Settings dialog, System Dep dialog, Help dialogs, Review/Issue dialogs
  - **widgets/**: Flow graph visualization (`flow_graph.py`), Icon utilities (`icons.py`)

### Key Paths

- **Tools storage**: `~/.cmdforge/<toolname>/config.yaml`
- **Tool defaults**: `~/.cmdforge/<toolname>/defaults.yaml` (optional, published with tool)
- **Tool settings**: `~/.cmdforge/<toolname>/settings.yaml` (user overrides, auto-created from defaults)
- **Wrapper scripts**: `~/.local/bin/<toolname>` (auto-generated bash scripts)
- **Provider config**: `~/.cmdforge/providers.yaml`
- **Collections storage**: `~/.cmdforge/collections/<name>.yaml`

### Tool Structure

Tools are YAML configs with:
- `name`, `description`, `category`
- `arguments`: Custom flags with defaults (e.g., `--max` → `{max}`)
- `steps`: Ordered list of `prompt` or `code` steps
- `output`: Template for final output (e.g., `"{response}"`)

### Step Types

1. **Prompt Step**: Calls AI provider with template, stores result in `output_var`
   - `profile`: AI persona (system prompt prefix)
   - `strip_fences`: Remove markdown code fences from output
   - `structured_output`: JSON schema for validated structured responses
2. **Code Step**: Executes Python code via `exec()`, captures specified variables (comma-separated for multiple outputs)
3. **Tool Step**: Calls another tool (meta-tools), supports `args` dict and `provider` override. Dependencies resolved via `resolver.py`

### Variable Flow

Variables are passed between steps:
- `{input}` - always available (stdin/file content)
- `{argname}` - from tool arguments
- `{step_output_var}` - from previous step's `output_var`
- `{settings.key}` - from tool's settings.yaml (top-level scalars only in templates)
- `settings['key']` - full dict access in code steps

Variable substitution is handled in `runner.py:substitute_variables()`. Settings are loaded from `settings.yaml` if it exists, otherwise an empty dict is used.

## Provider System

Providers are CLI tools that accept prompts via stdin and output to stdout. Defined in `~/.cmdforge/providers.yaml`:

```yaml
providers:
  - name: claude
    command: "claude -p"
  - name: mock
    command: "echo '[MOCK]'"
```

The `mock` provider is built-in for testing without API calls. Use `--provider mock` or `--dry-run` flags when testing tools.

## Web UI & Registry

CmdForge includes a web interface and tool registry:

### Web Modules (`src/cmdforge/web/`)

- **app.py**: Flask app factory, registers blueprints
- **routes.py**: Main web routes (docs, tutorials, tools, etc.)
- **auth.py**: Authentication middleware and decorators
- **sessions.py**: Session management
- **email.py**: Email utilities (password reset, notifications)
- **seo.py**: SEO utilities (sitemap, meta tags)
- **filters.py**: Jinja2 filters (timeago, markdown, etc.)
- **docs_content.py**: Documentation and tutorial content
- **forum/**: Community forum blueprint
  - **models.py**: Forum database schema (categories, topics, replies)
  - **routes.py**: Forum routes (/forum, /forum/c/<slug>, /forum/t/<id>)

### Registry Modules (`src/cmdforge/registry/`)

- **app.py**: Flask-based Registry API (tool publishing, search, downloads, authentication, rate limiting)
- **db.py**: SQLite schema and queries (`connect_db()`, `query_one()`, `query_all()`)
- **embeddings.py**: Semantic search with vector embeddings for tool discovery
- **sync.py**: Git-based tool sync from Gitea repository
- **categorize.py**: Auto-categorization of tools based on content
- **rate_limit.py**: Rate limiting for API endpoints
- **scrutiny.py**: Tool vetting (honesty, transparency, scope, efficiency checks)
- **similarity.py**: Duplicate detection via embedding similarity
- **stats.py**: Registry statistics and metrics
- **settings.py**: Server-side settings management

### Key URLs

- `/forum` - Community forum
- `/docs` - Documentation
- `/tutorials` - Tutorial guides
- `/tools` - Tool registry browser

### Running the Web UI

```bash
# Development
python -m cmdforge.web.app

# Production (example)
CMDFORGE_REGISTRY_DB=/path/to/db PORT=5050 python -m cmdforge.web.app
```

## Testing Conventions

Tests use `pytest` without a shared conftest.py. Common patterns:

**Mocking strategy:**
- **File system**: Use `tmp_path` fixture and patch module-level paths (`TOOLS_DIR`, `BIN_DIR`, `PROVIDERS_FILE`)
- **Subprocess calls**: Mock `subprocess.run` and `shutil.which`
- **Provider calls**: Mock `call_provider` to return `ProviderResult(text="...", success=True)`
- **Registry calls**: Mock `requests` for API tests

**Example fixture pattern:**
```python
@pytest.fixture
def temp_providers_file(tmp_path):
    providers_file = tmp_path / "providers.yaml"
    with patch('cmdforge.providers.PROVIDERS_FILE', providers_file):
        yield providers_file
```

Integration tests are marked with `@pytest.mark.integration` and require a running registry server.

## Infrastructure Documentation

For deployment, server details, and operations, see the `docs/` folder:

- **docs/servers.md** - Server IPs (192.168.0.162), SSH access, paths, service commands
- **docs/deployment.md** - Architecture diagram, deploy process, systemd service config
- **docs/maintenance.md** - Backups, updates, troubleshooting
- **docs/architecture.md** - Module structure, data flow diagrams

### Production Server Quick Reference

| Property | Value |
|----------|-------|
| Server | 192.168.0.162 (OpenMediaVault) |
| SSH | `ssh rob@192.168.0.162` |
| App Path | `/srv/mergerfs/data_pool/home/rob/cmdforge-registry/` |
| Service | `systemctl --user status cmdforge-web` |
| Public URL | https://cmdforge.brrd.tech |
| Port | 5050 (via Cloudflare tunnel) |