Move docs to project-docs symlink structure
- Docs now point to project-docs/docs/projects/cmdforge - Old docs preserved in olddocs/ - Updated .gitignore to ignore docs symlink 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
This commit is contained in:
Claude Opus 4.5
@@ -0,0 +1,116 @@
|
||||
# CmdForge Collections
|
||||
|
||||
Collections are curated groups of tools that can be installed together with a single command.
|
||||
|
||||
## Use Cases
|
||||
|
||||
1. **Thematic bundles**: "writing-toolkit" with grammar, tone, simplify tools
|
||||
2. **Application stacks**: "data-science" with json-extract, csv-insights, etc.
|
||||
3. **Source bundles**: "fabric-text" with all Fabric text processing patterns
|
||||
4. **Workflow packages**: Tools that work well together for a specific task
|
||||
|
||||
## Collection Manifest Format
|
||||
|
||||
Collections are defined in the registry repo under `collections/`:
|
||||
|
||||
```yaml
|
||||
# collections/writing-toolkit.yaml
|
||||
name: writing-toolkit
|
||||
display_name: Writing Toolkit
|
||||
description: Essential tools for writers and editors
|
||||
icon: pencil # Optional icon identifier
|
||||
maintainer: official # Publisher who maintains this collection
|
||||
|
||||
tools:
|
||||
- official/fix-grammar
|
||||
- official/simplify
|
||||
- official/tone-shift
|
||||
- official/expand
|
||||
- official/proofread
|
||||
|
||||
# Optional: version constraints
|
||||
pinned:
|
||||
official/fix-grammar: "^1.0.0"
|
||||
|
||||
# Optional: suggested order for documentation
|
||||
order:
|
||||
- official/fix-grammar
|
||||
- official/proofread
|
||||
- official/simplify
|
||||
- official/tone-shift
|
||||
- official/expand
|
||||
|
||||
tags:
|
||||
- writing
|
||||
- editing
|
||||
- grammar
|
||||
```
|
||||
|
||||
## CLI Usage
|
||||
|
||||
```bash
|
||||
# List available collections
|
||||
cmdforge collections list
|
||||
|
||||
# View collection details
|
||||
cmdforge collections info writing-toolkit
|
||||
|
||||
# Install all tools in a collection
|
||||
cmdforge collections install writing-toolkit
|
||||
|
||||
# Install with version constraints from collection
|
||||
cmdforge collections install writing-toolkit --pinned
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
```
|
||||
GET /api/v1/collections # List all collections
|
||||
GET /api/v1/collections/:name # Get collection details with tool info
|
||||
GET /api/v1/collections/:name/tools # Get just the tools list
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE collections (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT UNIQUE NOT NULL,
|
||||
display_name TEXT NOT NULL,
|
||||
description TEXT,
|
||||
icon TEXT,
|
||||
maintainer TEXT NOT NULL,
|
||||
tools TEXT NOT NULL, -- JSON array of tool refs
|
||||
pinned TEXT, -- JSON object of version constraints
|
||||
tags TEXT, -- JSON array
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE INDEX idx_collections_name ON collections(name);
|
||||
CREATE INDEX idx_collections_maintainer ON collections(maintainer);
|
||||
```
|
||||
|
||||
## Web UI
|
||||
|
||||
- `/collections` - Browse all collections
|
||||
- `/collections/:name` - Collection detail page with tool grid
|
||||
- Install button that shows CLI command
|
||||
- Filter by tag, maintainer
|
||||
|
||||
## Sync from Registry Repo
|
||||
|
||||
Collections sync from the registry repo just like tools:
|
||||
|
||||
```
|
||||
CmdForge-Registry/
|
||||
├── tools/
|
||||
│ └── ...
|
||||
└── collections/
|
||||
├── writing-toolkit.yaml
|
||||
├── data-science.yaml
|
||||
├── fabric-text.yaml
|
||||
└── fabric-code.yaml
|
||||
```
|
||||
|
||||
The webhook sync process reads `collections/*.yaml` and upserts to database.
|
||||
@@ -0,0 +1,264 @@
|
||||
# CmdForge Deployment Guide
|
||||
|
||||
This guide covers deploying the CmdForge Registry web application.
|
||||
|
||||
## Quick Start (Development)
|
||||
|
||||
```bash
|
||||
# Clone and install
|
||||
git clone https://gitea.brrd.tech/rob/CmdForge.git
|
||||
cd CmdForge
|
||||
pip install -e ".[dev]"
|
||||
|
||||
# Initialize database
|
||||
cd src/cmdforge/web
|
||||
python -c "from app import create_app; create_app()"
|
||||
|
||||
# Run development server
|
||||
flask run --port 5050 --host 0.0.0.0
|
||||
```
|
||||
|
||||
Access at `http://localhost:5050`
|
||||
|
||||
## Production Deployment
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.8+
|
||||
- pip/pipx
|
||||
- nginx (recommended for reverse proxy)
|
||||
- systemd (for process management)
|
||||
|
||||
### 1. Install Application
|
||||
|
||||
```bash
|
||||
# Create application directory
|
||||
sudo mkdir -p /opt/cmdforge
|
||||
sudo chown $USER:$USER /opt/cmdforge
|
||||
cd /opt/cmdforge
|
||||
|
||||
# Clone repository
|
||||
git clone https://gitea.brrd.tech/rob/CmdForge.git .
|
||||
|
||||
# Create virtual environment
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install dependencies
|
||||
pip install -e ".[prod]"
|
||||
pip install gunicorn
|
||||
```
|
||||
|
||||
### 2. Configure Environment
|
||||
|
||||
Create `/opt/cmdforge/.env`:
|
||||
|
||||
```bash
|
||||
# Required
|
||||
FLASK_ENV=production
|
||||
SECRET_KEY=your-secret-key-here # Generate with: python -c "import secrets; print(secrets.token_hex(32))"
|
||||
|
||||
# Optional
|
||||
SENTRY_DSN=https://[email protected]/xxx # Error monitoring
|
||||
SITE_URL=https://cmdforge.brrd.tech # For sitemap/SEO
|
||||
```
|
||||
|
||||
### 3. Initialize Database
|
||||
|
||||
```bash
|
||||
cd /opt/cmdforge/src/cmdforge/web
|
||||
python -c "from app import create_app; create_app()"
|
||||
```
|
||||
|
||||
Database will be created at `data/cmdforge.db`.
|
||||
|
||||
### 4. systemd Service
|
||||
|
||||
Create `/etc/systemd/system/cmdforge-web.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=CmdForge Registry Web Application
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=www-data
|
||||
Group=www-data
|
||||
WorkingDirectory=/opt/cmdforge
|
||||
Environment="PATH=/opt/cmdforge/venv/bin"
|
||||
EnvironmentFile=/opt/cmdforge/.env
|
||||
ExecStart=/opt/cmdforge/venv/bin/gunicorn \
|
||||
--workers 4 \
|
||||
--bind 127.0.0.1:5050 \
|
||||
--access-logfile /var/log/cmdforge/access.log \
|
||||
--error-logfile /var/log/cmdforge/error.log \
|
||||
'cmdforge.web.app:create_app()'
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Enable and start:
|
||||
|
||||
```bash
|
||||
# Create log directory
|
||||
sudo mkdir -p /var/log/cmdforge
|
||||
sudo chown www-data:www-data /var/log/cmdforge
|
||||
|
||||
# Enable service
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable cmdforge-web
|
||||
sudo systemctl start cmdforge-web
|
||||
|
||||
# Check status
|
||||
sudo systemctl status cmdforge-web
|
||||
```
|
||||
|
||||
### 5. nginx Configuration
|
||||
|
||||
Create `/etc/nginx/sites-available/cmdforge`:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name cmdforge.brrd.tech;
|
||||
|
||||
# Redirect HTTP to HTTPS
|
||||
return 301 https://$server_name$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name cmdforge.brrd.tech;
|
||||
|
||||
# SSL certificates (use certbot for Let's Encrypt)
|
||||
ssl_certificate /etc/letsencrypt/live/cmdforge.brrd.tech/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/cmdforge.brrd.tech/privkey.pem;
|
||||
|
||||
# SSL settings
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
ssl_prefer_server_ciphers on;
|
||||
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
|
||||
|
||||
# Static files (optional - Flask can serve these)
|
||||
location /static/ {
|
||||
alias /opt/cmdforge/src/cmdforge/web/static/;
|
||||
expires 1d;
|
||||
add_header Cache-Control "public, immutable";
|
||||
}
|
||||
|
||||
# Proxy to gunicorn
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:5050;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Enable site:
|
||||
|
||||
```bash
|
||||
sudo ln -s /etc/nginx/sites-available/cmdforge /etc/nginx/sites-enabled/
|
||||
sudo nginx -t
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
### 6. SSL with Let's Encrypt
|
||||
|
||||
```bash
|
||||
sudo apt install certbot python3-certbot-nginx
|
||||
sudo certbot --nginx -d cmdforge.brrd.tech
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Required | Description |
|
||||
|----------|----------|-------------|
|
||||
| `FLASK_ENV` | Yes | `production` or `development` |
|
||||
| `SECRET_KEY` | Yes | Secret key for sessions (32+ chars) |
|
||||
| `SENTRY_DSN` | No | Sentry error monitoring DSN |
|
||||
| `SITE_URL` | No | Public URL for sitemap generation |
|
||||
| `DATABASE_PATH` | No | Override database path (default: `data/cmdforge.db`) |
|
||||
|
||||
## Database Backup
|
||||
|
||||
```bash
|
||||
# Backup
|
||||
cp /opt/cmdforge/data/cmdforge.db /backup/cmdforge-$(date +%Y%m%d).db
|
||||
|
||||
# Or use SQLite backup command for consistency
|
||||
sqlite3 /opt/cmdforge/data/cmdforge.db ".backup '/backup/cmdforge.db'"
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Logs
|
||||
|
||||
```bash
|
||||
# Application logs
|
||||
tail -f /var/log/cmdforge/error.log
|
||||
tail -f /var/log/cmdforge/access.log
|
||||
|
||||
# systemd logs
|
||||
journalctl -u cmdforge-web -f
|
||||
```
|
||||
|
||||
### Sentry Integration
|
||||
|
||||
The application includes Sentry integration for error monitoring. Set `SENTRY_DSN` environment variable to enable.
|
||||
|
||||
### Health Check
|
||||
|
||||
```bash
|
||||
curl http://localhost:5050/api/v1/tools
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Service won't start
|
||||
|
||||
```bash
|
||||
# Check logs
|
||||
journalctl -u cmdforge-web -n 50
|
||||
|
||||
# Verify Python path
|
||||
/opt/cmdforge/venv/bin/python -c "import cmdforge"
|
||||
|
||||
# Test gunicorn manually
|
||||
cd /opt/cmdforge
|
||||
source venv/bin/activate
|
||||
gunicorn --bind 127.0.0.1:5050 'cmdforge.web.app:create_app()'
|
||||
```
|
||||
|
||||
### Database errors
|
||||
|
||||
```bash
|
||||
# Check database exists and is readable
|
||||
ls -la /opt/cmdforge/data/cmdforge.db
|
||||
|
||||
# Verify permissions
|
||||
chown www-data:www-data /opt/cmdforge/data/
|
||||
chown www-data:www-data /opt/cmdforge/data/cmdforge.db
|
||||
```
|
||||
|
||||
### Static files not loading
|
||||
|
||||
```bash
|
||||
# Rebuild Tailwind CSS
|
||||
cd /opt/cmdforge
|
||||
npm install
|
||||
npm run css:build
|
||||
|
||||
# Or use development CSS
|
||||
# (static/css/output.css should exist)
|
||||
```
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
See `docs/diagrams/deployment-architecture.svg` for visual representation.
|
||||
@@ -0,0 +1,425 @@
|
||||
# CmdForge - Design Document
|
||||
|
||||
> A lightweight personal tool builder for AI-powered CLI commands
|
||||
|
||||
## Overview
|
||||
|
||||
CmdForge lets you create custom AI-powered terminal commands. You define a tool once (name, steps, provider), then use it like any Linux command.
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
# Create a summarizer tool, then use it:
|
||||
sum -i text.txt -o summary.txt --max 512
|
||||
```
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Tool = Directory + Config
|
||||
|
||||
```
|
||||
~/.cmdforge/
|
||||
sum/
|
||||
config.yaml
|
||||
processed.py # Optional external code file
|
||||
reviewer/
|
||||
config.yaml
|
||||
translator/
|
||||
config.yaml
|
||||
```
|
||||
|
||||
### config.yaml Format
|
||||
|
||||
```yaml
|
||||
name: sum
|
||||
description: "Summarize documents"
|
||||
arguments:
|
||||
- flag: --max
|
||||
variable: max
|
||||
default: "500"
|
||||
description: "Maximum words in summary"
|
||||
steps:
|
||||
- type: prompt
|
||||
prompt: |
|
||||
Summarize the following text in {max} words or less:
|
||||
|
||||
{input}
|
||||
provider: claude
|
||||
output_var: response
|
||||
output: "{response}"
|
||||
|
||||
# Optional: declare dependencies for meta-tools
|
||||
dependencies:
|
||||
- official/summarize
|
||||
- official/translate@^1.0.0 # With version constraint
|
||||
```
|
||||
|
||||
### Step Types
|
||||
|
||||
**Prompt Step** - Calls an AI provider:
|
||||
```yaml
|
||||
- type: prompt
|
||||
prompt: "Your prompt template with {variables}"
|
||||
provider: claude
|
||||
output_var: response
|
||||
```
|
||||
|
||||
**Code Step** - Runs Python code:
|
||||
```yaml
|
||||
- type: code
|
||||
code: |
|
||||
processed = input.upper()
|
||||
count = len(processed.split())
|
||||
output_var: processed, count
|
||||
code_file: processed.py # Optional: external file storage
|
||||
```
|
||||
|
||||
**Tool Step** - Calls another CmdForge tool (meta-tools):
|
||||
```yaml
|
||||
- type: tool
|
||||
tool: official/summarize # Tool reference (owner/name or just name for local)
|
||||
input: "{input}" # Input to pass (supports variable substitution)
|
||||
args: # Optional arguments
|
||||
max_words: "100"
|
||||
output_var: summary # Variable to store the tool's output
|
||||
provider: claude # Optional: override the called tool's provider
|
||||
```
|
||||
|
||||
Steps execute in order. Each step's `output_var` becomes available to subsequent steps.
|
||||
|
||||
### Variables
|
||||
|
||||
- `{input}` - Always available, contains stdin or input file content (empty string if no input)
|
||||
- `{variable_name}` - From arguments (e.g., `{max}`)
|
||||
- `{output_var}` - From previous steps (e.g., `{response}`, `{processed}`)
|
||||
|
||||
### Output Variables
|
||||
|
||||
The `output_var` field specifies which Python variable(s) to capture from your code:
|
||||
|
||||
**Single variable:**
|
||||
```yaml
|
||||
output_var: processed
|
||||
```
|
||||
```python
|
||||
processed = input.upper() # This gets captured
|
||||
```
|
||||
|
||||
**Multiple variables (comma-separated):**
|
||||
```yaml
|
||||
output_var: processed, count, summary
|
||||
```
|
||||
```python
|
||||
processed = input.upper()
|
||||
count = len(processed.split())
|
||||
summary = f"Processed {count} words"
|
||||
# All three are captured and available as {processed}, {count}, {summary}
|
||||
```
|
||||
|
||||
## CLI Interface
|
||||
|
||||
### Running Tools
|
||||
|
||||
```bash
|
||||
# Basic usage (wrapper script in ~/.local/bin)
|
||||
sum -i document.txt -o summary.txt
|
||||
|
||||
# With custom args
|
||||
sum -i document.txt --max 200
|
||||
|
||||
# Preview prompt without calling AI
|
||||
sum -i document.txt --dry-run
|
||||
|
||||
# See what prompt gets sent
|
||||
sum -i document.txt --show-prompt
|
||||
|
||||
# Test with mock (no API call)
|
||||
sum -i document.txt --provider mock
|
||||
|
||||
# Read from stdin, write to stdout
|
||||
cat doc.txt | sum | less
|
||||
|
||||
# Interactive stdin input
|
||||
sum --stdin
|
||||
|
||||
# No input (empty string) - useful for tools using only arguments
|
||||
sum --max 100
|
||||
|
||||
# Or via cmdforge run
|
||||
cmdforge run sum -i document.txt
|
||||
```
|
||||
|
||||
### Input Handling
|
||||
|
||||
| Scenario | Behavior |
|
||||
|----------|----------|
|
||||
| Piped stdin | Automatically read (`cat file.txt \| mytool`) |
|
||||
| `-i file.txt` | Read from file |
|
||||
| `--stdin` | Interactive input (type then Ctrl+D) |
|
||||
| No input | Empty string (useful for argument-only tools) |
|
||||
|
||||
### Universal Flags (all tools)
|
||||
|
||||
| Flag | Short | Description |
|
||||
|------|-------|-------------|
|
||||
| `--input` | `-i` | Input file |
|
||||
| `--output` | `-o` | Output file (or stdout if omitted) |
|
||||
| `--stdin` | | Read input interactively (type then Ctrl+D) |
|
||||
| `--dry-run` | | Show prompt, don't call AI |
|
||||
| `--show-prompt` | | Call AI but also print prompt to stderr |
|
||||
| `--provider` | `-p` | Override provider (e.g., `--provider mock`) |
|
||||
| `--verbose` | `-v` | Show debug info |
|
||||
| `--help` | `-h` | Show help |
|
||||
|
||||
### Managing Tools
|
||||
|
||||
```bash
|
||||
# Launch the UI to manage tools
|
||||
cmdforge
|
||||
|
||||
# Or use CLI directly:
|
||||
cmdforge list # List all tools
|
||||
cmdforge create sum # Create new tool (basic)
|
||||
cmdforge edit sum # Edit tool config in $EDITOR
|
||||
cmdforge delete sum # Delete tool
|
||||
cmdforge test sum # Test with mock provider
|
||||
cmdforge run sum # Run tool for real
|
||||
cmdforge refresh # Refresh all wrapper scripts
|
||||
cmdforge check sum # Check dependencies for meta-tools
|
||||
cmdforge ui # Launch interactive UI
|
||||
```
|
||||
|
||||
## Terminal UI
|
||||
|
||||
A BIOS-style terminal UI using `urwid` with full mouse support.
|
||||
|
||||
### Main Menu
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────┐
|
||||
│ CmdForge Manager │
|
||||
├──────────────────────────────────────┤
|
||||
│ Tools: │
|
||||
│ ┌────────────────────────────────┐ │
|
||||
│ │ sum │ │
|
||||
│ │ reviewer │ │
|
||||
│ │ translator │ │
|
||||
│ └────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Create] [Edit] [Delete] [Test] │
|
||||
│ │
|
||||
│ Tool Info: │
|
||||
│ Name: sum │
|
||||
│ Description: Summarize documents │
|
||||
│ Arguments: --max │
|
||||
│ Steps: PROMPT[claude] -> {response} │
|
||||
│ │
|
||||
│ [EXIT] │
|
||||
└──────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Tool Builder
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ New Tool │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ ┌─[ Tool Info ]──────────┐ ┌─ Arguments ─────────────────┐ │
|
||||
│ │ Name: [sum__________] │ │ --max -> {max} │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ Desc: [Summarize____] │ └─────────────────────────────┘ │
|
||||
│ │ │ [Add] [Edit] [Delete] │
|
||||
│ │ Output: [{response}_] │ │
|
||||
│ └────────────────────────┘ ┌─ Execution Steps ───────────┐ │
|
||||
│ │ P:claude -> {response} │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────┘ │
|
||||
│ [Add] [Edit] [Delete] │
|
||||
│ │
|
||||
│ [Save] [Cancel] │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Navigation:**
|
||||
- **Tab** - Cycle between sections (Tool Info -> Arguments -> Steps -> Buttons)
|
||||
- **Arrow keys** - Navigate within lists
|
||||
- **Enter/Click** - Select/activate
|
||||
- **Escape** - Cancel/back
|
||||
- **Shift + mouse** - Terminal-native text selection (bypasses UI mouse handling)
|
||||
- **Ctrl+Shift+V** - Paste
|
||||
|
||||
The current section's title is highlighted with brackets: `[ Tool Info ]`
|
||||
|
||||
**Copy/Paste tip:** Hold `Shift` while dragging with mouse to select text, then use your terminal's copy function (usually right-click or Ctrl+Shift+C).
|
||||
|
||||
### Code Step Dialog
|
||||
|
||||
```
|
||||
┌──────────────── Add Code Step ─────────────────┐
|
||||
│ Variables: input, max, response │
|
||||
│ │
|
||||
│ File: [processed.py______________] < Load > │
|
||||
│ │
|
||||
│ ┌─ Code ─────────────────────────────────────┐ │
|
||||
│ │processed = input.upper() │ │
|
||||
│ │count = len(processed.split()) │ │
|
||||
│ │ │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Output var: [processed, count____] │
|
||||
├────────────────────────────────────────────────┤
|
||||
│ < OK > < Cancel > │
|
||||
└────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Code Step Features:**
|
||||
- **Multiline editor** - Write multi-line Python code
|
||||
- **External file storage** - Code is auto-saved to `~/.cmdforge/<toolname>/<filename>` on OK
|
||||
- **Load button** - Load code from external file (with confirmation)
|
||||
- **Multiple output vars** - Capture multiple variables (comma-separated)
|
||||
|
||||
## Implementation
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
cmdforge/
|
||||
__init__.py
|
||||
cli.py # Entry point, argument parsing
|
||||
ui.py # UI dispatcher (chooses urwid/snack/dialog)
|
||||
ui_urwid.py # Urwid-based BIOS-style UI
|
||||
ui_snack.py # Snack/newt fallback UI
|
||||
tool.py # Tool loading/saving/wrapper scripts
|
||||
runner.py # Execute tools
|
||||
providers.py # Provider abstraction
|
||||
```
|
||||
|
||||
### Provider System
|
||||
|
||||
Providers are defined in `~/.cmdforge/providers.yaml`:
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- name: claude
|
||||
command: "claude -p"
|
||||
description: "Anthropic Claude"
|
||||
- name: codex
|
||||
command: "codex -p"
|
||||
description: "OpenAI Codex"
|
||||
- name: gemini
|
||||
command: "gemini"
|
||||
description: "Google Gemini"
|
||||
```
|
||||
|
||||
The provider command receives the prompt via stdin and outputs to stdout.
|
||||
|
||||
### Wrapper Scripts
|
||||
|
||||
When you save a tool, CmdForge creates a wrapper script in `~/.local/bin/`:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# CmdForge wrapper for 'sum'
|
||||
# Auto-generated - do not edit
|
||||
exec /path/to/python -m cmdforge.runner sum "$@"
|
||||
```
|
||||
|
||||
The wrapper uses the full Python path to ensure the correct environment is used.
|
||||
|
||||
Use `cmdforge refresh` to regenerate all wrapper scripts (e.g., after changing Python environments).
|
||||
|
||||
### Tool Name Validation
|
||||
|
||||
Tool names must:
|
||||
- Start with a letter or underscore
|
||||
- Contain only letters, numbers, underscores, and dashes
|
||||
- Not contain spaces or shell-problematic characters (`/\|&;$`"'<>(){}[]!?*#~`)
|
||||
|
||||
## Tool Composition
|
||||
|
||||
CmdForge are designed to chain together like any Unix command.
|
||||
|
||||
### External Pipelines (Tool-to-Tool)
|
||||
|
||||
```bash
|
||||
# Chain multiple CmdForge
|
||||
cat logs.txt | log-errors | summarize | translate --lang Japanese
|
||||
|
||||
# Mix with standard Unix tools
|
||||
git log --oneline | head -20 | changelog | tee CHANGELOG.md
|
||||
|
||||
# Build complex workflows
|
||||
cat *.py | review-code --focus security | json-extract --fields "issue, severity, file" | json2csv
|
||||
```
|
||||
|
||||
Each tool reads stdin and writes stdout. No special integration needed.
|
||||
|
||||
### Internal Pipelines (Multi-Step)
|
||||
|
||||
Within a single tool, chain steps for preprocessing/validation:
|
||||
|
||||
```yaml
|
||||
steps:
|
||||
- type: code # Preprocess
|
||||
code: |
|
||||
filtered = '\n'.join(l for l in input.split('\n') if 'ERROR' in l)
|
||||
output_var: filtered
|
||||
- type: prompt # AI processes filtered input
|
||||
prompt: "Explain these errors: {filtered}"
|
||||
provider: claude-haiku
|
||||
output_var: explanation
|
||||
- type: code # Post-process
|
||||
code: |
|
||||
result = f"Found {len(filtered.split(chr(10)))} errors:\n\n{explanation}"
|
||||
output_var: result
|
||||
output: "{result}"
|
||||
```
|
||||
|
||||
**When to use which:**
|
||||
- **External pipelines**: Reusable tools, different providers per stage, standard Unix interop
|
||||
- **Internal pipelines**: Tightly coupled steps, shared context, validation of AI output
|
||||
|
||||
## What This Design Doesn't Include
|
||||
|
||||
Intentionally omitted (not needed for personal use):
|
||||
|
||||
- Trust tiers / security levels
|
||||
- Cryptographic signing
|
||||
- Container isolation / sandboxing
|
||||
- Certification testing
|
||||
- Distribution packaging
|
||||
- PII redaction
|
||||
- Audit logging
|
||||
- Provider capability negotiation
|
||||
|
||||
**Why?** This is a personal tool builder. You write the tools, you run the tools, you accept the responsibility. Just like any bash script you write.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Required:
|
||||
- Python 3.10+
|
||||
- PyYAML
|
||||
- urwid (for TUI)
|
||||
|
||||
Optional fallbacks:
|
||||
- python3-newt/snack (simpler TUI)
|
||||
- dialog/whiptail (basic TUI)
|
||||
|
||||
## Example Workflow
|
||||
|
||||
1. Run `cmdforge` to open UI
|
||||
2. Select "Create" to create a new tool
|
||||
3. Fill in: name, description, output template
|
||||
4. Add arguments (e.g., `--max` with default `500`)
|
||||
5. Add a prompt step with your prompt template and provider
|
||||
6. Click "Save"
|
||||
7. Exit UI
|
||||
8. Run `sum -i myfile.txt -o summary.txt`
|
||||
|
||||
Done. No containers, no signing, no certification. Just a tool that works.
|
||||
|
||||
## Default Values
|
||||
|
||||
| Step Type | Default Output Var | Default File |
|
||||
|-----------|-------------------|--------------|
|
||||
| Prompt | `response` | n/a |
|
||||
| Code | `processed` | `processed.py` |
|
||||
+1058
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,249 @@
|
||||
# Installation Guide
|
||||
|
||||
Complete installation instructions for CmdForge.
|
||||
|
||||
## Quick Install
|
||||
|
||||
```bash
|
||||
git clone https://gitea.brrd.tech/rob/CmdForge.git
|
||||
cd CmdForge
|
||||
pip install -e .
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
cmdforge providers install
|
||||
```
|
||||
|
||||
## Detailed Installation
|
||||
|
||||
### Step 1: Install Python 3.10+
|
||||
|
||||
**Ubuntu/Debian:**
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install python3 python3-pip
|
||||
```
|
||||
|
||||
**macOS:**
|
||||
```bash
|
||||
brew install [email protected]
|
||||
```
|
||||
|
||||
**Verify:**
|
||||
```bash
|
||||
python3 --version # Should be 3.10 or higher
|
||||
```
|
||||
|
||||
### Step 2: Install CmdForge
|
||||
|
||||
```bash
|
||||
git clone https://gitea.brrd.tech/rob/CmdForge.git
|
||||
cd CmdForge
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**With TUI and development dependencies:**
|
||||
```bash
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
### Step 3: Configure PATH
|
||||
|
||||
CmdForge creates executable wrappers in `~/.local/bin/`. Add this to your shell config:
|
||||
|
||||
**Bash (~/.bashrc):**
|
||||
```bash
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
```
|
||||
|
||||
**Zsh (~/.zshrc):**
|
||||
```bash
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
```
|
||||
|
||||
**Fish (~/.config/fish/config.fish):**
|
||||
```fish
|
||||
set -gx PATH $HOME/.local/bin $PATH
|
||||
```
|
||||
|
||||
Then reload:
|
||||
```bash
|
||||
source ~/.bashrc # or restart terminal
|
||||
```
|
||||
|
||||
### Step 4: Install an AI Provider
|
||||
|
||||
You need at least one AI CLI tool. The easiest way is the interactive installer:
|
||||
|
||||
```bash
|
||||
cmdforge providers install
|
||||
```
|
||||
|
||||
This guides you through installing and authenticating providers.
|
||||
|
||||
#### Manual Installation
|
||||
|
||||
**OpenCode (free tier available):**
|
||||
```bash
|
||||
curl -fsSL https://opencode.ai/install | bash
|
||||
source ~/.bashrc # or start new shell
|
||||
opencode # Opens browser for sign-in
|
||||
```
|
||||
|
||||
**Claude CLI (pay-per-use):**
|
||||
```bash
|
||||
npm install -g @anthropic-ai/claude-code
|
||||
claude # Opens browser for sign-in
|
||||
```
|
||||
|
||||
**Codex (pay-per-use):**
|
||||
```bash
|
||||
npm install -g @openai/codex
|
||||
codex # Opens browser for sign-in
|
||||
```
|
||||
|
||||
**Gemini (free tier available):**
|
||||
```bash
|
||||
npm install -g @google/gemini-cli
|
||||
gemini # Opens browser for Google sign-in
|
||||
```
|
||||
|
||||
**Ollama (free, runs locally):**
|
||||
```bash
|
||||
curl -fsSL https://ollama.ai/install.sh | bash
|
||||
ollama pull llama3
|
||||
cmdforge providers add ollama "ollama run llama3" -d "Local Llama 3"
|
||||
```
|
||||
|
||||
See [PROVIDERS.md](PROVIDERS.md) for full provider setup instructions.
|
||||
|
||||
### Step 5: Verify Installation
|
||||
|
||||
```bash
|
||||
# Check CmdForge is installed
|
||||
cmdforge --version
|
||||
|
||||
# List configured providers
|
||||
cmdforge providers list
|
||||
|
||||
# Check which providers are available
|
||||
cmdforge providers check
|
||||
|
||||
# Test with mock provider (no AI needed)
|
||||
echo "hello world" | cmdforge run --provider mock
|
||||
```
|
||||
|
||||
### Step 6: Create Wrapper Scripts
|
||||
|
||||
After installation or any Python environment change:
|
||||
|
||||
```bash
|
||||
cmdforge refresh
|
||||
```
|
||||
|
||||
This creates executable scripts in `~/.local/bin/` for all your tools.
|
||||
|
||||
## Installing Example Tools
|
||||
|
||||
### Quick Install (All 28 Tools)
|
||||
|
||||
```bash
|
||||
python examples/install.py
|
||||
cmdforge refresh
|
||||
```
|
||||
|
||||
### Manual Install
|
||||
|
||||
Copy individual tool configs to `~/.cmdforge/<toolname>/config.yaml`.
|
||||
|
||||
See [EXAMPLES.md](EXAMPLES.md) for all tool configurations.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "cmdforge: command not found"
|
||||
|
||||
PATH is not configured. Add to your shell config:
|
||||
```bash
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
```
|
||||
|
||||
### "Provider 'X' not found"
|
||||
|
||||
1. Check provider is installed: `which claude` or `which opencode`
|
||||
2. Check provider is configured: `cmdforge providers`
|
||||
3. Add missing provider: `cmdforge providers add`
|
||||
|
||||
### "Permission denied" on tool execution
|
||||
|
||||
Refresh wrapper scripts:
|
||||
```bash
|
||||
cmdforge refresh
|
||||
```
|
||||
|
||||
### TUI doesn't start / looks broken
|
||||
|
||||
Install urwid:
|
||||
```bash
|
||||
pip install urwid
|
||||
```
|
||||
|
||||
Ensure your terminal supports mouse (most modern terminals do).
|
||||
|
||||
### Tools work with `cmdforge run` but not directly
|
||||
|
||||
The wrapper scripts may be outdated. Refresh them:
|
||||
```bash
|
||||
cmdforge refresh
|
||||
```
|
||||
|
||||
### Python version mismatch
|
||||
|
||||
If you have multiple Python versions, ensure CmdForge uses the right one:
|
||||
```bash
|
||||
python3.11 -m pip install cmdforge
|
||||
```
|
||||
|
||||
## Uninstalling
|
||||
|
||||
```bash
|
||||
# Remove package
|
||||
pip uninstall cmdforge
|
||||
|
||||
# Remove config and tools (optional)
|
||||
rm -rf ~/.cmdforge
|
||||
|
||||
# Remove wrapper scripts (optional)
|
||||
rm ~/.local/bin/{summarize,translate,fix-grammar,...}
|
||||
```
|
||||
|
||||
## Upgrading
|
||||
|
||||
```bash
|
||||
pip install --upgrade cmdforge
|
||||
cmdforge refresh
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
After installation:
|
||||
|
||||
```
|
||||
~/.cmdforge/
|
||||
├── providers.yaml # Provider configurations
|
||||
├── summarize/
|
||||
│ └── config.yaml # Tool config
|
||||
├── translate/
|
||||
│ ├── config.yaml
|
||||
│ └── prompt.txt # External prompt file (optional)
|
||||
└── ...
|
||||
|
||||
~/.local/bin/
|
||||
├── cmdforge # Main command
|
||||
├── summarize # Tool wrapper script
|
||||
├── translate # Tool wrapper script
|
||||
└── ...
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. [Set up providers](PROVIDERS.md)
|
||||
2. [Browse example tools](EXAMPLES.md)
|
||||
3. Run `cmdforge ui` to create your first tool
|
||||
@@ -0,0 +1,249 @@
|
||||
# Meta-Tools: Tools That Call Other Tools
|
||||
|
||||
Meta-tools are CmdForge tools that can invoke other tools as steps in their workflow. This enables powerful composition and reuse of existing tools.
|
||||
|
||||
## Step Type: `tool`
|
||||
|
||||
A new step type `tool` allows calling another CmdForge tool from within a tool's workflow.
|
||||
|
||||
### Manifest Format
|
||||
|
||||
```yaml
|
||||
name: summarize-and-translate
|
||||
description: Summarize text then translate the summary
|
||||
version: 1.0.0
|
||||
category: Text
|
||||
|
||||
steps:
|
||||
- type: tool
|
||||
tool: official/summarize # Tool to call (owner/name or just name for local)
|
||||
input: "{input}" # What to pass as input (supports variable substitution)
|
||||
args: # Optional arguments to pass to the tool
|
||||
max_words: "100"
|
||||
output_var: summary # Variable to store the output
|
||||
|
||||
- type: tool
|
||||
tool: official/translate
|
||||
input: "{summary}" # Use output from previous step
|
||||
args:
|
||||
target_language: "{language}" # Can use tool's own arguments
|
||||
output_var: translated
|
||||
|
||||
output: "{translated}"
|
||||
|
||||
arguments:
|
||||
- flag: "--language"
|
||||
variable: language
|
||||
default: "Spanish"
|
||||
description: "Target language for translation"
|
||||
```
|
||||
|
||||
### Resolution Order
|
||||
|
||||
When resolving a tool reference in a `tool` step:
|
||||
|
||||
1. **Fully qualified**: `owner/name` - looks up in registry or installed tools
|
||||
2. **Local tool**: Just `name` - checks local `~/.cmdforge/<name>/config.yaml` first
|
||||
3. **Registry fallback**: If not found locally, checks installed registry tools
|
||||
|
||||
### Tool Step Properties
|
||||
|
||||
| Property | Required | Description |
|
||||
|----------|----------|-------------|
|
||||
| `type` | Yes | Must be `"tool"` |
|
||||
| `tool` | Yes | Tool reference (owner/name or name) |
|
||||
| `input` | No | Input to pass to the tool (default: current `{input}`) |
|
||||
| `args` | No | Dictionary of arguments to pass |
|
||||
| `output_var` | Yes | Variable name to store the tool's output |
|
||||
| `provider` | No | Override provider for the called tool |
|
||||
|
||||
### Execution
|
||||
|
||||
When a `tool` step executes:
|
||||
|
||||
1. Resolve the tool reference to find the tool definition
|
||||
2. Substitute variables in `input` and `args` values
|
||||
3. Execute the tool with the resolved input and arguments
|
||||
4. Capture the output in the specified `output_var`
|
||||
|
||||
### Error Handling
|
||||
|
||||
- If a called tool fails, the parent tool fails with an appropriate error
|
||||
- Tool not found errors include helpful installation suggestions
|
||||
- Circular dependencies are detected and prevented
|
||||
|
||||
## Dependency Resolution
|
||||
|
||||
### Declaring Dependencies
|
||||
|
||||
Tools can declare their dependencies in the config:
|
||||
|
||||
```yaml
|
||||
name: my-meta-tool
|
||||
dependencies:
|
||||
- official/summarize
|
||||
- official/translate@^1.0.0 # With version constraint
|
||||
```
|
||||
|
||||
### CLI Integration
|
||||
|
||||
```bash
|
||||
# Install a tool and its dependencies
|
||||
cmdforge install official/summarize-and-translate
|
||||
|
||||
# Check if dependencies are satisfied
|
||||
cmdforge check my-meta-tool
|
||||
|
||||
# Install missing dependencies
|
||||
cmdforge install --deps my-meta-tool
|
||||
```
|
||||
|
||||
### Dependency Checking
|
||||
|
||||
Before running a meta-tool, the runner verifies all dependencies are installed:
|
||||
|
||||
```python
|
||||
def check_dependencies(tool: Tool) -> List[str]:
|
||||
"""Returns list of missing dependencies."""
|
||||
missing = []
|
||||
for dep in tool.dependencies:
|
||||
if not is_tool_installed(dep):
|
||||
missing.append(dep)
|
||||
return missing
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Sandboxing**: Called tools run in the same process context as the parent
|
||||
2. **Input validation**: All inputs are treated as untrusted strings
|
||||
3. **No shell execution**: Tool references cannot contain shell commands
|
||||
4. **Depth limit**: Maximum nesting depth (default: 10) prevents runaway recursion
|
||||
|
||||
## Example: Multi-Step Analysis Pipeline
|
||||
|
||||
```yaml
|
||||
name: code-review-pipeline
|
||||
description: Comprehensive code review using multiple specialized tools
|
||||
version: 1.0.0
|
||||
category: Developer
|
||||
|
||||
dependencies:
|
||||
- official/analyze-complexity
|
||||
- official/find-bugs
|
||||
- official/suggest-improvements
|
||||
|
||||
steps:
|
||||
# Step 1: Analyze code complexity
|
||||
- type: tool
|
||||
tool: official/analyze-complexity
|
||||
input: "{input}"
|
||||
output_var: complexity_report
|
||||
|
||||
# Step 2: Find potential bugs
|
||||
- type: tool
|
||||
tool: official/find-bugs
|
||||
input: "{input}"
|
||||
output_var: bug_report
|
||||
|
||||
# Step 3: Generate improvement suggestions based on findings
|
||||
- type: tool
|
||||
tool: official/suggest-improvements
|
||||
input: |
|
||||
## Code:
|
||||
{input}
|
||||
|
||||
## Complexity Analysis:
|
||||
{complexity_report}
|
||||
|
||||
## Bug Report:
|
||||
{bug_report}
|
||||
output_var: suggestions
|
||||
|
||||
output: |
|
||||
# Code Review Results
|
||||
|
||||
## Complexity Analysis
|
||||
{complexity_report}
|
||||
|
||||
## Potential Issues
|
||||
{bug_report}
|
||||
|
||||
## Improvement Suggestions
|
||||
{suggestions}
|
||||
```
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### ToolStep Dataclass
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class ToolStep:
|
||||
"""A step that calls another tool."""
|
||||
tool: str # Tool reference (owner/name or name)
|
||||
output_var: str # Variable to store output
|
||||
input_template: str = "{input}" # Input template
|
||||
args: Dict[str, str] = field(default_factory=dict)
|
||||
provider: Optional[str] = None # Provider override
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
d = {
|
||||
"type": "tool",
|
||||
"tool": self.tool,
|
||||
"output_var": self.output_var,
|
||||
}
|
||||
if self.input_template != "{input}":
|
||||
d["input"] = self.input_template
|
||||
if self.args:
|
||||
d["args"] = self.args
|
||||
if self.provider:
|
||||
d["provider"] = self.provider
|
||||
return d
|
||||
```
|
||||
|
||||
### Execution Function
|
||||
|
||||
```python
|
||||
def execute_tool_step(
|
||||
step: ToolStep,
|
||||
variables: dict,
|
||||
depth: int = 0,
|
||||
max_depth: int = 10
|
||||
) -> tuple[str, bool]:
|
||||
"""Execute a tool step by calling another tool."""
|
||||
if depth >= max_depth:
|
||||
return "", False # Prevent infinite recursion
|
||||
|
||||
# Resolve the tool
|
||||
try:
|
||||
resolved = resolve_tool(step.tool)
|
||||
except ToolNotFoundError:
|
||||
print(f"Error: Tool '{step.tool}' not found", file=sys.stderr)
|
||||
return "", False
|
||||
|
||||
# Prepare input
|
||||
input_text = substitute_variables(step.input_template, variables)
|
||||
|
||||
# Prepare arguments
|
||||
args = {}
|
||||
for key, value in step.args.items():
|
||||
args[key] = substitute_variables(value, variables)
|
||||
|
||||
# Run the tool
|
||||
output, exit_code = run_tool(
|
||||
tool=resolved.tool,
|
||||
input_text=input_text,
|
||||
custom_args=args,
|
||||
provider_override=step.provider,
|
||||
_depth=depth + 1 # Track nesting depth
|
||||
)
|
||||
|
||||
return output, exit_code == 0
|
||||
```
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- **Parallel tool execution**: Run independent tool steps concurrently
|
||||
- **Conditional execution**: Skip steps based on conditions
|
||||
- **Tool aliases**: Define shorthand names for frequently used tools
|
||||
- **Caching**: Cache tool outputs for identical inputs
|
||||
@@ -0,0 +1,180 @@
|
||||
# CmdForge Project Overview
|
||||
|
||||
This document provides a comprehensive overview of the CmdForge project structure, components, and how everything fits together.
|
||||
|
||||
## What is CmdForge?
|
||||
|
||||
CmdForge is a personal tool builder for AI-powered CLI commands. It consists of three main components:
|
||||
|
||||
1. **CLI Tool** (`cmdforge`) - Create, manage, and run AI-powered command-line tools
|
||||
2. **Registry API** - REST API for publishing and discovering tools
|
||||
3. **Web UI** - Browser interface for the registry with docs, search, and user dashboard
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Component | Location | Purpose |
|
||||
|-----------|----------|---------|
|
||||
| CLI | `src/cmdforge/cli.py` | Main entry point for `cmdforge` command |
|
||||
| Registry API | `src/cmdforge/registry/` | REST API for tool registry |
|
||||
| Web UI | `src/cmdforge/web/` | Flask web application |
|
||||
| Tool Storage | `~/.cmdforge/` | User's installed tools |
|
||||
| Database | `data/cmdforge.db` | SQLite with FTS5 search |
|
||||
|
||||
## Architecture Diagrams
|
||||
|
||||
See `docs/diagrams/` for visual representations:
|
||||
|
||||
- **system-architecture.svg** - How components interact
|
||||
- **deployment-architecture.svg** - Production deployment setup
|
||||
- **source-organization.svg** - Code structure
|
||||
|
||||
## Source Code Structure
|
||||
|
||||
```
|
||||
src/cmdforge/
|
||||
├── cli.py # Entry point, subcommand routing
|
||||
├── tool.py # Tool dataclasses, YAML loading
|
||||
├── runner.py # Step execution engine
|
||||
├── providers.py # AI provider abstraction
|
||||
├── config.py # Global configuration
|
||||
├── manifest.py # Tool manifest handling
|
||||
│
|
||||
├── registry/ # Registry API Server
|
||||
│ ├── app.py # Flask API application
|
||||
│ ├── db.py # Database operations, FTS5 search
|
||||
│ ├── rate_limit.py # API rate limiting
|
||||
│ ├── sync.py # Git sync for tool submissions
|
||||
│ ├── categorize.py # Auto-categorization
|
||||
│ └── similarity.py # Similar tool recommendations
|
||||
│
|
||||
├── web/ # Web UI Application
|
||||
│ ├── app.py # Flask web app, Sentry integration
|
||||
│ ├── routes.py # Page routes and handlers
|
||||
│ ├── auth.py # Password hashing, login/register
|
||||
│ ├── sessions.py # Session management
|
||||
│ ├── docs_content.py # Documentation text content
|
||||
│ ├── seo.py # Sitemap, robots.txt
|
||||
│ ├── filters.py # Jinja template filters
|
||||
│ ├── templates/ # Jinja HTML templates
|
||||
│ └── static/ # CSS, JS, images
|
||||
│
|
||||
├── registry_client.py # CLI client for registry API
|
||||
├── resolver.py # Dependency resolution
|
||||
│
|
||||
├── ui.py # TUI dispatcher
|
||||
├── ui_urwid.py # urwid TUI implementation
|
||||
└── ui_snack.py # newt/snack TUI fallback
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
SQLite database at `data/cmdforge.db`:
|
||||
|
||||
| Table | Purpose |
|
||||
|-------|---------|
|
||||
| `publishers` | User accounts (username, email, password_hash) |
|
||||
| `tools` | Published tools (name, owner, description, config) |
|
||||
| `api_tokens` | Authentication tokens for API access |
|
||||
| `download_stats` | Tool download tracking |
|
||||
| `pageviews` | Analytics for tool detail pages |
|
||||
| `tools_fts` | Full-text search index (FTS5) |
|
||||
|
||||
## API Endpoints
|
||||
|
||||
Base URL: `/api/v1/`
|
||||
|
||||
| Endpoint | Method | Purpose |
|
||||
|----------|--------|---------|
|
||||
| `/tools` | GET | List/search tools |
|
||||
| `/tools/<owner>/<name>` | GET | Get tool details |
|
||||
| `/tools/<owner>/<name>` | POST | Publish tool (auth required) |
|
||||
| `/tools/<owner>/<name>/download` | POST | Record download |
|
||||
| `/me` | GET | Current user info |
|
||||
| `/tokens` | GET/POST | Manage API tokens |
|
||||
|
||||
## Web Routes
|
||||
|
||||
| Route | Purpose |
|
||||
|-------|---------|
|
||||
| `/` | Homepage with featured tools |
|
||||
| `/tools` | Tool browser with search |
|
||||
| `/tools/<owner>/<name>` | Tool detail page |
|
||||
| `/docs/*` | Documentation pages |
|
||||
| `/login`, `/register` | Authentication |
|
||||
| `/dashboard/*` | User dashboard, token management |
|
||||
|
||||
## Key Files to Know
|
||||
|
||||
### Configuration
|
||||
- `~/.cmdforge/providers.yaml` - AI provider configurations
|
||||
- `~/.cmdforge/config.yaml` - Global settings
|
||||
- `~/.cmdforge/<tool>/config.yaml` - Individual tool configs
|
||||
|
||||
### Development
|
||||
- `pyproject.toml` - Package configuration
|
||||
- `pytest.ini` - Test configuration
|
||||
- `tailwind.config.js` - Tailwind CSS configuration
|
||||
|
||||
### Documentation
|
||||
- `docs/REGISTRY.md` - Registry design document (comprehensive)
|
||||
- `docs/WEB_UI.md` - Web UI design document (comprehensive)
|
||||
- `wiki/Home.md` - User-facing wiki documentation
|
||||
- `CLAUDE.md` - Development guidance for AI assistants
|
||||
|
||||
## Running the Project
|
||||
|
||||
### Development
|
||||
|
||||
```bash
|
||||
# Install with dev dependencies
|
||||
pip install -e ".[dev]"
|
||||
|
||||
# Run CLI
|
||||
python -m cmdforge.cli
|
||||
|
||||
# Run web server (development)
|
||||
cd src/cmdforge/web
|
||||
flask run --port 5050
|
||||
```
|
||||
|
||||
### Production
|
||||
|
||||
```bash
|
||||
# Using gunicorn
|
||||
gunicorn -w 4 -b 0.0.0.0:5050 'cmdforge.web.app:create_app()'
|
||||
|
||||
# Environment variables needed:
|
||||
# FLASK_ENV=production
|
||||
# SECRET_KEY=<your-secret-key>
|
||||
# SENTRY_DSN=<optional-sentry-dsn>
|
||||
# SITE_URL=https://your-domain.com
|
||||
```
|
||||
|
||||
### systemd Service
|
||||
|
||||
See `docs/DEPLOYMENT.md` for systemd service configuration.
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
pytest
|
||||
|
||||
# Run specific test
|
||||
pytest tests/test_name.py::test_function
|
||||
|
||||
# Run with coverage
|
||||
pytest --cov=cmdforge
|
||||
```
|
||||
|
||||
## Design Documents
|
||||
|
||||
For detailed design decisions and specifications:
|
||||
|
||||
- **docs/REGISTRY.md** - Complete registry API design (Phases 1-8)
|
||||
- **docs/WEB_UI.md** - Complete web UI design (Phase 7)
|
||||
- **wiki/Home.md** - User documentation for CLI usage
|
||||
|
||||
## Archive
|
||||
|
||||
Old development discussions and diagrams are preserved in `archive/discussions/` for historical reference.
|
||||
@@ -0,0 +1,328 @@
|
||||
# Provider Setup Guide
|
||||
|
||||
CmdForge works with any AI CLI tool that accepts input via stdin or arguments. This guide covers setup for the most popular providers.
|
||||
|
||||
## Provider Comparison
|
||||
|
||||
We profiled 12 providers with a 4-task benchmark (Math, Code, Reasoning, Data Extraction):
|
||||
|
||||
| Provider | Speed | Score | Cost | Best For |
|
||||
|----------|-------|-------|------|----------|
|
||||
| **opencode-deepseek** | 13s | 4/4 | ~$0.28/M tokens | **Best value** - daily driver |
|
||||
| **opencode-pickle** | 13s | 4/4 | FREE | **Best free** - accurate |
|
||||
| **claude-haiku** | 14s | 4/4 | ~$0.25/M tokens | Fast + high quality |
|
||||
| **codex** | 14s | 4/4 | ~$1.25/M tokens | Reliable, auto-routes |
|
||||
| **claude** | 18s | 4/4 | Varies | Auto-routes to best |
|
||||
| **claude-opus** | 18s | 4/4 | ~$15/M tokens | Highest quality |
|
||||
| **claude-sonnet** | 21s | 4/4 | ~$3/M tokens | Balanced |
|
||||
| **opencode-nano** | 24s | 4/4 | Paid | GPT-5 Nano |
|
||||
| **gemini-flash** | 28s | 4/4 | ~$0.075/M tokens | Google, faster |
|
||||
| **opencode-reasoner** | 33s | 4/4 | ~$0.28/M tokens | Complex reasoning |
|
||||
| **gemini** | 91s | 3/4 | ~$1.25/M tokens | 1M token context |
|
||||
| **opencode-grok** | 11s | 2/4 | FREE | Fastest but unreliable |
|
||||
|
||||
### Recommendations
|
||||
|
||||
- **Daily use:** `opencode-deepseek` or `opencode-pickle` (free)
|
||||
- **Quality work:** `claude-haiku` or `claude-opus`
|
||||
- **Complex reasoning:** `opencode-reasoner`
|
||||
- **Large documents:** `gemini` (1M token context window)
|
||||
- **Budget:** `opencode-pickle` (free) or `opencode-deepseek` (cheap)
|
||||
|
||||
## Provider Setup
|
||||
|
||||
### OpenCode (Recommended)
|
||||
|
||||
OpenCode provides access to multiple models including free options.
|
||||
|
||||
**Install:**
|
||||
```bash
|
||||
curl -fsSL https://opencode.ai/install | bash
|
||||
```
|
||||
|
||||
**Authenticate:**
|
||||
```bash
|
||||
~/.opencode/bin/opencode auth
|
||||
```
|
||||
|
||||
**Available Models:**
|
||||
| Provider Name | Model | Cost |
|
||||
|---------------|-------|------|
|
||||
| `opencode-deepseek` | deepseek-chat | Cheap |
|
||||
| `opencode-pickle` | big-pickle | FREE |
|
||||
| `opencode-grok` | grok-code | FREE |
|
||||
| `opencode-nano` | gpt-5-nano | Paid |
|
||||
| `opencode-reasoner` | deepseek-reasoner | Cheap |
|
||||
|
||||
**Test:**
|
||||
```bash
|
||||
echo "Hello" | ~/.opencode/bin/opencode run --model opencode/big-pickle
|
||||
```
|
||||
|
||||
### Claude CLI
|
||||
|
||||
Anthropic's official CLI for Claude models.
|
||||
|
||||
**Install:**
|
||||
```bash
|
||||
npm install -g @anthropic-ai/claude-code
|
||||
```
|
||||
|
||||
**Authenticate:**
|
||||
```bash
|
||||
claude # Opens browser for sign-in (auto-saves auth tokens)
|
||||
```
|
||||
|
||||
**Available Models:**
|
||||
| Provider Name | Model | Cost |
|
||||
|---------------|-------|------|
|
||||
| `claude` | Auto-routes | Varies |
|
||||
| `claude-haiku` | Haiku 4.5 | Cheap |
|
||||
| `claude-sonnet` | Sonnet 4.5 | Medium |
|
||||
| `claude-opus` | Opus 4.5 | Expensive |
|
||||
|
||||
**Test:**
|
||||
```bash
|
||||
echo "Hello" | claude -p
|
||||
```
|
||||
|
||||
### Codex (OpenAI)
|
||||
|
||||
OpenAI's Codex CLI with auto-routing.
|
||||
|
||||
**Install:**
|
||||
```bash
|
||||
npm install -g @openai/codex
|
||||
```
|
||||
|
||||
**Authenticate:**
|
||||
```bash
|
||||
codex # Opens browser for sign-in (auto-saves auth tokens)
|
||||
```
|
||||
|
||||
**Test:**
|
||||
```bash
|
||||
echo "Hello" | codex exec -
|
||||
```
|
||||
|
||||
### Gemini
|
||||
|
||||
Google's Gemini models. Best for large context (1M tokens).
|
||||
|
||||
**Install:**
|
||||
```bash
|
||||
npm install -g @google/gemini-cli
|
||||
```
|
||||
|
||||
**Authenticate:**
|
||||
```bash
|
||||
gemini # Opens browser for Google sign-in
|
||||
```
|
||||
|
||||
**Available Models:**
|
||||
| Provider Name | Model | Notes |
|
||||
|---------------|-------|-------|
|
||||
| `gemini` | gemini-2.5-pro | Quality, slow CLI |
|
||||
| `gemini-flash` | gemini-2.5-flash | Faster |
|
||||
|
||||
**Note:** Gemini CLI has known performance issues. Use `gemini-flash` for interactive tasks, `gemini` for large documents.
|
||||
|
||||
**Test:**
|
||||
```bash
|
||||
echo "Hello" | gemini --model gemini-2.5-flash
|
||||
```
|
||||
|
||||
## Managing Providers
|
||||
|
||||
### Interactive Installation (Recommended)
|
||||
|
||||
The easiest way to install providers:
|
||||
|
||||
```bash
|
||||
cmdforge providers install
|
||||
```
|
||||
|
||||
This interactive guide:
|
||||
- Shows available AI providers with costs
|
||||
- Runs the installation command
|
||||
- Updates PATH automatically
|
||||
- Shows next steps for authentication
|
||||
|
||||
### List Providers
|
||||
|
||||
```bash
|
||||
cmdforge providers list
|
||||
```
|
||||
|
||||
### Check Availability
|
||||
|
||||
```bash
|
||||
cmdforge providers check
|
||||
```
|
||||
|
||||
### Add Custom Provider
|
||||
|
||||
```bash
|
||||
cmdforge providers add myname "my-command --args" -d "Description"
|
||||
```
|
||||
|
||||
Or edit `~/.cmdforge/providers.yaml`:
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- name: my-custom
|
||||
command: my-ai-tool --prompt
|
||||
description: My custom AI tool
|
||||
```
|
||||
|
||||
### Provider Command Format
|
||||
|
||||
The command should:
|
||||
1. Accept input via stdin
|
||||
2. Output response to stdout
|
||||
3. Exit 0 on success
|
||||
|
||||
Example commands:
|
||||
```bash
|
||||
# Claude
|
||||
claude -p
|
||||
|
||||
# OpenCode
|
||||
$HOME/.opencode/bin/opencode run --model deepseek/deepseek-chat
|
||||
|
||||
# Gemini
|
||||
gemini --model gemini-2.5-flash
|
||||
|
||||
# Codex
|
||||
codex exec -
|
||||
|
||||
# Custom (any tool that reads stdin)
|
||||
my-tool --input -
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Provider commands can use environment variables:
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- name: opencode
|
||||
command: $HOME/.opencode/bin/opencode run
|
||||
```
|
||||
|
||||
`$HOME` and `~` are expanded automatically.
|
||||
|
||||
## Using Providers in Tools
|
||||
|
||||
### In Tool Config
|
||||
|
||||
```yaml
|
||||
steps:
|
||||
- type: prompt
|
||||
prompt: "Summarize: {input}"
|
||||
provider: opencode-pickle # Use this provider
|
||||
output_var: response
|
||||
```
|
||||
|
||||
### Override at Runtime
|
||||
|
||||
```bash
|
||||
# Use a different provider for this run
|
||||
cat file.txt | summarize --provider claude-opus
|
||||
```
|
||||
|
||||
### Provider Selection Strategy
|
||||
|
||||
1. **Tool default** - Set in tool's config.yaml
|
||||
2. **Runtime override** - `--provider` flag
|
||||
3. **Cost optimization** - Use cheap providers for simple tasks
|
||||
4. **Quality needs** - Use opus/sonnet for important work
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Provider 'X' not found"
|
||||
|
||||
1. Check it's in your providers list: `cmdforge providers`
|
||||
2. Verify the command works: `echo "test" | <command>`
|
||||
3. Add it: `cmdforge providers add`
|
||||
|
||||
### "Command 'X' not found"
|
||||
|
||||
The AI CLI tool isn't installed or not in PATH:
|
||||
```bash
|
||||
which claude # Should show path
|
||||
which opencode # Might need full path
|
||||
```
|
||||
|
||||
For OpenCode, use full path in provider:
|
||||
```yaml
|
||||
command: $HOME/.opencode/bin/opencode run
|
||||
```
|
||||
|
||||
### Slow Provider
|
||||
|
||||
- Use `gemini-flash` instead of `gemini`
|
||||
- Use `claude-haiku` instead of `claude-opus`
|
||||
- Use `opencode-deepseek` for best speed/quality ratio
|
||||
|
||||
### Provider Errors
|
||||
|
||||
Check the provider works directly:
|
||||
```bash
|
||||
echo "Say hello" | claude -p
|
||||
echo "Say hello" | ~/.opencode/bin/opencode run
|
||||
```
|
||||
|
||||
If it works directly but not in CmdForge, check:
|
||||
1. Provider command in `~/.cmdforge/providers.yaml`
|
||||
2. Environment variables are expanded correctly
|
||||
|
||||
## Cost Optimization
|
||||
|
||||
### Free Providers
|
||||
|
||||
- `opencode-pickle` - Big Pickle model (FREE, accurate)
|
||||
- `opencode-grok` - Grok Code (FREE, fast but less reliable)
|
||||
|
||||
### Cheap Providers
|
||||
|
||||
- `opencode-deepseek` - ~$0.28/M tokens
|
||||
- `opencode-reasoner` - ~$0.28/M tokens
|
||||
- `claude-haiku` - ~$0.25/M tokens
|
||||
|
||||
### Tips
|
||||
|
||||
1. Use `opencode-pickle` for simple tasks (free + accurate)
|
||||
2. Use `claude-haiku` when you need reliability
|
||||
3. Reserve `claude-opus` for important work
|
||||
4. Use `gemini` only for large document analysis
|
||||
|
||||
## Adding New Providers
|
||||
|
||||
Any CLI tool that:
|
||||
- Reads from stdin
|
||||
- Writes to stdout
|
||||
- Exits 0 on success
|
||||
|
||||
Can be a provider. Examples:
|
||||
|
||||
**Local LLM (Ollama):**
|
||||
```yaml
|
||||
- name: ollama-llama
|
||||
command: ollama run llama3
|
||||
description: Local Llama 3
|
||||
```
|
||||
|
||||
**Custom API wrapper:**
|
||||
```yaml
|
||||
- name: my-api
|
||||
command: curl -s -X POST https://my-api.com/chat -d @-
|
||||
description: My custom API
|
||||
```
|
||||
|
||||
**Python script:**
|
||||
```yaml
|
||||
- name: my-python
|
||||
command: python3 ~/scripts/my_ai.py
|
||||
description: Custom Python AI
|
||||
```
|
||||
+2023
File diff suppressed because it is too large
Load Diff
+1462
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user