rob
/
CmdForge
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
CmdForge/CLAUDE.md

14 KiB
Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

CmdForge is a lightweight personal tool builder for AI-powered CLI commands. It lets users create custom terminal commands that call AI providers, chain prompts with Python code steps, and use them like any Unix pipe command.

Development Commands

# Install for development
pip install -e ".[dev]"

# Run all unit tests (excluding integration tests that need a server)
pytest tests/ -m "not integration"

# Run a specific test file
pytest tests/test_runner.py -v

# Run a specific test class or method
pytest tests/test_runner.py::TestSubstituteVariables -v
pytest tests/test_runner.py::TestSubstituteVariables::test_simple_substitution -v

# Run with coverage
pytest tests/ --cov=cmdforge --cov-report=html

# Run integration tests (requires local registry server at localhost:5000)
python -m cmdforge.registry.app  # Start server first
pytest tests/test_registry_integration.py -v -m integration

# CLI entry points
cmdforge              # Main CLI / GUI launcher
cf                    # Interactive tool picker (fzf-style)
python -m cmdforge.cli  # Alternative CLI invocation

Architecture

Core Modules (src/cmdforge/)

  • cli/: CLI commands entry points (cmdforge command). Routes subcommands: list, create, edit, delete, test, run, ui, docs, check, refresh, providers, registry, collections, deps, install, lock, verify, add, remove, init, config, settings, system-deps
  • tool.py: Tool definition dataclasses (Tool, ToolArgument, PromptStep, CodeStep, ToolStep, McpStep), YAML config loading/saving, wrapper script generation
  • runner.py: Execution engine. Runs tool steps sequentially, handles variable substitution ({input}, {varname}), executes Python code steps via exec(), handles nested tool calls with depth limit (MAX_TOOL_DEPTH=10)
  • resolver.py: Tool resolution. resolve_tool() searches: project manifest → local tools → owner/name → registry. Returns ResolvedTool with path info
  • collection.py: Collection management (Collection dataclass, resolve_tool_references(), classify_tool_reference()), local collection storage in ~/.cmdforge/collections/
  • providers.py: Provider abstraction. Supports subprocess CLI tools, OpenAI-compatible HTTP APIs, experimental PTY wrappers, fallback chains, and tool/MCP-server allowlists. Auto-discovers installed providers on first run. Config in ~/.cmdforge/providers.yaml with versioned migration.
  • skills.py: Validated Agent Skills loader for per-provider SKILL.md context under ~/.cmdforge/providers/<name>/skills/
  • mcp_client.py / mcp_server.py: Stdio and Streamable HTTP MCP integration, configuration, schema discovery, transport security, and exposure policy
  • profiles.py: AI persona profiles with system prompts, stored in ~/.cmdforge/profiles/
  • manifest.py: Project manifest (cmdforge.yaml) for declaring tool dependencies with version constraints
  • lockfile.py: Lock file support for reproducible installs (cmdforge.lock)
  • registry_client.py: Client for registry API (search, publish, download, authentication)
  • config.py: Global configuration management (registry URL, auth tokens)
  • system_deps.py: System package dependency management (apt/dnf/pacman detection and installation)
  • dependency_graph.py: Dependency graph resolution for meta-tools
  • hash_utils.py: Hash verification utilities for tool integrity
  • gui/: PySide6 desktop GUI
    • main_window.py: Main application window with sidebar navigation
    • pages/: Welcome page, Tools page, Tool Builder, Registry browser, Collections page, Providers management, Profiles
    • dialogs/: Step editors (prompt/code/tool), Argument editor, Provider dialog, Provider Install dialog, Connect/Publish dialogs, Test Step dialog, Settings dialog, System Dep dialog, Help dialogs, Review/Issue dialogs
    • widgets/: Flow graph visualization (flow_graph.py), Icon utilities (icons.py)

Key Paths

  • Tools storage: ~/.cmdforge/<toolname>/config.yaml
  • Tool defaults: ~/.cmdforge/<toolname>/defaults.yaml (optional, published with tool)
  • Tool settings: ~/.cmdforge/<toolname>/settings.yaml (user overrides, auto-created from defaults)
  • Wrapper scripts: ~/.local/bin/<toolname> (auto-generated bash scripts)
  • Provider config: ~/.cmdforge/providers.yaml
  • Collections storage: ~/.cmdforge/collections/<name>.yaml

Tool Structure

Tools are YAML configs with:

  • name, description, category
  • arguments: Custom flags with defaults (e.g., --max → {max})
  • steps: Ordered list of prompt, code, or tool steps
  • output: Template for final output (e.g., "{response}")
  • input_schema, output_schema: Optional tool-level JSON Schema contracts; schemas are validated when tools are loaded or created
  • deprecated, deprecated_message, replacement: Validated migration metadata; replacements may be local names or owner/tool registry references

Run cmdforge inspect <tool> [--registry] for the shared local preflight report. cmdforge registry publish <path> --dry-run runs local checks first and, when authenticated, the registry's publish-time checks without publishing. Normal publication uses the same checks as Stage 1, asks for confirmation in an interactive terminal, and then performs the Stage 2 mutation. The registry reruns deterministic conformance authoritatively and stores immutable audit and quality evidence for the published version. Inspect also shows inferred contract proposals and runs deterministic contract conformance when both tool schemas are explicit. Prompt outputs are synthesized from their schemas; code, nested-tool, and MCP steps are reported as unsupported instead of being executed implicitly. Use cmdforge inspect <tool> --save-baseline to explicitly promote passing evidence to conformance.json. Later inspections and local publish dry-runs rerun the stored synthetic inputs and report state, coverage, contract, and normalized output changes. ToolStep preflight also compares the values the runner supplies with the called tool's input contract. Quality headlines exclude categories with missing evidence and are accompanied by evidence coverage plus checked/not-tested category states.

Agent-facing discovery

Use cmdforge list --json --filter "<need>" --limit 10 for a compact local catalog containing names, descriptions, argument definitions, and contracts. Complete definitions (including prompt and code bodies) require an explicit --full. If no local tool fits, use cmdforge registry search "<need>" --json --limit 5. For one-off AI work, pipe data through cmdforge run-once "Instruction {input}"; repeated workflows should become normal versioned tools. Do not force CmdForge into application code where a direct SDK/API is itself the intended runtime dependency.

Step Types

  1. Prompt Step: Calls AI provider with template, stores result in output_var
    • profile: AI persona (system prompt prefix)
    • strip_fences: Remove markdown code fences from output
    • output_schema: JSON schema for validated structured responses
    • max_tokens: Max output tokens (provider-dependent, e.g., 4096 for haiku)
    • plain_text: Bypass structured output enforcement
  2. Code Step: Executes Python code via exec(), captures specified variables (comma-separated for multiple outputs)
  3. Tool Step: Calls another tool (meta-tools), supports input_template, args, and recursive agent context (provider, profile, skills, tools). Provider and delegated allowlists are both enforced before resolution.
  4. MCP Step: Calls a tool on a configured MCP server. Provider mcp_servers allowlists are enforced before connecting.

Variable Flow

Variables are passed between steps:

  • {input} - always available (stdin/file content)
  • {argname} - from tool arguments
  • {step_output_var} - from previous step's output_var
  • {settings.key} - from tool's settings.yaml (top-level scalars only in templates)
  • settings['key'] - full dict access in code steps

Variable substitution is handled in runner.py:substitute_variables(). Settings are loaded from settings.yaml if it exists, otherwise an empty dict is used.

Provider System

Providers wrap AI CLIs or compatible HTTP APIs. Defined in ~/.cmdforge/providers.yaml:

version: 2
providers:
  - name: claude
    command: "claude -p"
    description: "Anthropic Claude"
    type: subprocess
  - name: openrouter
    command: "https://openrouter.ai/api/v1"
    description: "OpenRouter API"
    type: api
    model: openrouter/auto-beta
    api_key_env: OPENROUTER_API_KEY
  - name: mock
    command: "mock"
    description: "Mock provider for testing"

Provider fields:

  • name: Provider identifier used in tool configs
  • command: Shell command (subprocess/pty) or endpoint URL (api)
  • type: subprocess (default), api (OpenAI-compatible HTTP), or pty (interactive CLI, experimental)
  • model: Model ID for api-type providers
  • api_key_env: Environment variable holding the API key for api-type providers
  • description: Optional human-readable description
  • fallback: Optional provider to try if this one fails
  • fallback_chain: Ordered list of providers for multi-step fallback
  • tags: List of strings (e.g. ["free", "code", "local"])
  • pty_config: Dict with prompt_pattern, response_pattern, exit_command for pty providers
  • install: Optional structured install metadata dict
  • tools: Optional CmdForge tool allowlist; null allows all and [] denies all
  • mcp_servers: Optional MCP server allowlist; null allows all and [] denies all

The mock provider is built-in for testing without API calls. Use --provider mock or --dry-run flags when testing tools.

Provider CLI commands:

  • cmdforge providers list - List all providers and their status
  • cmdforge providers check - Check which providers are available
  • cmdforge providers add <name> <command> - Add/update a provider (supports --type, --model, --api-key-env, --tag, --fallback, --fallback-chain)
  • cmdforge providers remove <name> - Remove a provider
  • cmdforge providers test <name> - Test a provider
  • cmdforge providers discover [--add] - Scan system for installed CLIs and API keys
  • cmdforge providers install - Interactive guide to install AI providers
  • cmdforge providers for-tools <tool> [tools...] - List providers used by tools (with --warm to pre-load local models)

Web UI & Registry

CmdForge includes a web interface and tool registry:

Web Modules (src/cmdforge/web/)

  • app.py: Flask app factory, registers blueprints
  • routes.py: Main web routes (docs, tutorials, tools, etc.)
  • auth.py: Authentication middleware and decorators
  • sessions.py: Session management
  • email.py: Email utilities (password reset, notifications)
  • seo.py: SEO utilities (sitemap, meta tags)
  • filters.py: Jinja2 filters (timeago, markdown, etc.)
  • docs_content.py: Documentation and tutorial content
  • forum/: Community forum blueprint
    • models.py: Forum database schema (categories, topics, replies)
    • routes.py: Forum routes (/forum, /forum/c/, /forum/t/)

Registry Modules (src/cmdforge/registry/)

  • app.py: Flask-based Registry API (tool publishing, search, downloads, authentication, rate limiting)
  • db.py: SQLite schema and queries (connect_db(), query_one(), query_all())
  • embeddings.py: Semantic search with vector embeddings for tool discovery
  • sync.py: Git-based tool sync from Gitea repository
  • categorize.py: Auto-categorization of tools based on content
  • rate_limit.py: Rate limiting for API endpoints
  • scrutiny.py: Tool vetting (honesty, transparency, scope, efficiency checks)
  • similarity.py: Duplicate detection via embedding similarity
  • stats.py: Registry statistics and metrics
  • settings.py: Server-side settings management

Key URLs

  • /forum - Community forum
  • /docs - Documentation
  • /tutorials - Tutorial guides
  • /tools - Tool registry browser

Running the Web UI

# Development
python -m cmdforge.web.app

# Production (example)
CMDFORGE_REGISTRY_DB=/path/to/db PORT=5050 python -m cmdforge.web.app

Testing Conventions

Tests use pytest without a shared conftest.py. Common patterns:

Mocking strategy:

  • File system: Use tmp_path fixture and patch module-level paths (TOOLS_DIR, BIN_DIR, PROVIDERS_FILE)
  • Subprocess calls: Mock subprocess.run and shutil.which
  • Provider calls: Mock call_provider to return ProviderResult(text="...", success=True)
  • Registry calls: Mock requests for API tests

Example fixture pattern:

@pytest.fixture
def temp_providers_file(tmp_path):
    providers_file = tmp_path / "providers.yaml"
    with patch('cmdforge.providers.PROVIDERS_FILE', providers_file):
        yield providers_file

Integration tests are marked with @pytest.mark.integration and require a running registry server.

Infrastructure Documentation

For deployment, server details, and operations, see the docs/ folder:

  • docs/servers.md - Server IPs (192.168.0.162), SSH access, paths, service commands
  • docs/deployment.md - Architecture diagram, deploy process, systemd service config
  • docs/maintenance.md - Backups, updates, troubleshooting
  • docs/architecture.md - Module structure, data flow diagrams

Production Server Quick Reference

Property Value
Server 192.168.0.162 (OpenMediaVault)
SSH ssh rob@192.168.0.162
App Path /srv/mergerfs/data_pool/home/rob/cmdforge-registry/
Service systemctl --user status cmdforge-web
Public URL https://cmdforge.brrd.tech
Port 5050 (via Cloudflare tunnel)