CmdForge/README.md

# CmdForge

**A lightweight personal tool builder for AI-powered CLI commands.**

Turn any AI model into a Unix-style pipe command. Build once, use forever.

```bash
# Fix grammar in any file
echo "teh cat sat on teh mat" | fix-grammar
# Output: The cat sat on the mat

# Explain errors instantly
cat error.log | explain-error

# Generate commit messages
git diff --staged | commit-msg

# Extract data as validated JSON
echo "Price $49.99, SKU ABC-123" | json-extract --fields "price, sku"
# Output: {"price": 49.99, "sku": "ABC-123"}
```

## Why CmdForge?

- **Unix Philosophy** - Tools that do one thing well, composable with pipes
- **Provider Agnostic** - Works with Claude, GPT, Gemini, DeepSeek, local models
- **No Lock-in** - Your tools are YAML files, your prompts are yours
- **Multi-step Pipelines** - Chain AI prompts with Python code for validation
- **Meta-tools** - Tools can call other tools for powerful composition
- **12+ Providers Profiled** - We tested them so you don't have to

## Try It Now (60 seconds)

See CmdForge in action without installing anything on your system:

```bash
# Pull the container
docker pull gitea.brrd.tech/rob/cmdforge:latest

# Run it
docker run -it --rm gitea.brrd.tech/rob/cmdforge bash

# Inside the container - install OpenCode (includes free AI models)
cmdforge providers install   # Select option 4, then 'y'
source ~/.bashrc

# Try it! This explains the README using the free Big Pickle model
cat README.md | eli5
```

That's it - you just used AI to explain itself. The `eli5` tool uses a free model by default, no sign-up required.

## Quick Start (Native Install)

For regular use, install natively:

```bash
# Clone and install
git clone https://gitea.brrd.tech/rob/CmdForge.git
cd CmdForge
pip install -e .

# Ensure ~/.local/bin is in PATH
export PATH="$HOME/.local/bin:$PATH"

# Install an AI provider (interactive guide)
cmdforge providers install

# Launch the GUI
cmdforge

# Or create your first tool via CLI
cmdforge create summarize
```

## Installation

### Native Install

```bash
git clone https://gitea.brrd.tech/rob/CmdForge.git
cd CmdForge
pip install -e .
```

### With Development Dependencies

```bash
git clone https://gitea.brrd.tech/rob/CmdForge.git
cd CmdForge
pip install -e ".[dev]"
```

### Requirements

- Python 3.10+
- At least one AI CLI tool installed (see [Provider Setup](docs/reference/providers.md))
- PySide6 (included automatically - requires display server on Linux)

### Post-Install

Add to your `~/.bashrc` or `~/.zshrc`:

```bash
export PATH="$HOME/.local/bin:$PATH"
```

Then refresh wrapper scripts:

```bash
cmdforge refresh
```

## Usage

### GUI Mode (Recommended for Beginners)

```bash
cmdforge
```

Opens the graphical interface where you can create and manage tools visually. Features include:
- **My Tools** - Browse, create, edit, and delete tools organized by category
- **Registry** - Search and install community tools from the CmdForge registry
- **Providers** - Manage AI provider configurations

### CLI Mode

```bash
# Tool Management
cmdforge list              # List all tools
cmdforge create mytool     # Create new tool
cmdforge edit mytool       # Edit in $EDITOR
cmdforge delete mytool     # Delete tool
cmdforge run mytool        # Run a tool
cmdforge test mytool       # Test with mock provider
cmdforge check mytool      # Check dependencies (meta-tools)
cmdforge refresh           # Update executable wrappers
cmdforge docs mytool       # View/create tool documentation

# Registry
cmdforge registry search "keyword"    # Search for tools
cmdforge registry install owner/tool  # Install a tool
cmdforge registry publish mytool      # Publish your tool
cmdforge registry status              # Check moderation status
cmdforge registry my-tools            # List your published tools

# Collections (curated tool bundles)
cmdforge collections list             # List registry collections
cmdforge collections list --local     # List your local collections
cmdforge collections info starter     # View collection details
cmdforge collections install starter  # Install all tools in a collection
cmdforge collections create my-tools  # Create local collection
cmdforge collections show my-tools    # View local collection
cmdforge collections add my-tools summarize      # Add tool to collection
cmdforge collections remove my-tools summarize   # Remove tool
cmdforge collections delete my-tools  # Delete local collection
cmdforge collections publish my-tools # Publish to registry
cmdforge collections status my-tools  # Check tool statuses

# Project Dependencies
cmdforge init                         # Create cmdforge.yaml manifest
cmdforge add owner/tool               # Add tool as dependency
cmdforge install                      # Install all dependencies
cmdforge deps                         # Check dependency status

# Configuration
cmdforge config show                  # Show current config
cmdforge config connect username      # Connect to registry account
cmdforge config disconnect            # Disconnect from registry
```

### Running Tools

Once created, tools work like any Unix command:

```bash
# Direct invocation
mytool -i input.txt -o output.txt

# Pipe input (most common)
cat file.txt | mytool

# With arguments
echo "hello" | translate --lang French

# Preview without calling AI
cat file.txt | mytool --dry-run

# Test with mock provider
cat file.txt | mytool --provider mock
```

### Composing Tools

CmdForge follows Unix philosophy: each tool does one thing well, and tools chain together for complex workflows.

```bash
# Chain CmdForge together
cat error.log | log-errors | summarize | translate --lang Spanish

# Code review pipeline: focus on changes, review, then summarize
git diff | diff-focus | review-code | tldr

# Extract data, convert format, analyze
cat report.pdf | json-extract --fields "revenue, costs, date" | json2csv | csv-insights

# Process text through multiple transformations
cat technical_doc.txt | simplify | fix-grammar | tone-shift --tone friendly

# Generate code, validate it, then explain what it does
echo "parse CSV and calculate averages" | code-validate | explain-code
```

**Mix with standard Unix tools:**

```bash
# Extract emails, deduplicate, count
cat inbox.txt | extract-emails | sort -u | wc -l

# Find errors in multiple logs, explain them
cat *.log | grep -i error | explain-error

# Generate changelog for specific author
git log --author="rob" --oneline | changelog > CHANGELOG.md

# Review only large files in a commit
git diff --stat | awk '$3 > 100 {print $1}' | xargs cat | review-code
```

**Build shell functions for common pipelines:**

```bash
# ~/.bashrc
quick-review() {
    git diff "$@" | diff-focus | review-code --focus "bugs and security" | tldr
}

translate-doc() {
    cat "$1" | summarize | translate --lang "$2"
}

# Usage: quick-review HEAD~3
# Usage: translate-doc manual.txt French
```

## Example Tools

CmdForge comes with 28 pre-built examples you can install:

### Text Processing

| Tool | Description | Example |
|------|-------------|---------|
| `summarize` | Condense documents | `cat article.txt \| summarize` |
| `translate` | Translate text | `echo "Hello" \| translate --lang Spanish` |
| `fix-grammar` | Fix spelling/grammar | `cat draft.txt \| fix-grammar` |
| `simplify` | Rewrite for clarity | `cat legal.txt \| simplify --level "5th grade"` |
| `tone-shift` | Change tone | `cat email.txt \| tone-shift --tone professional` |
| `eli5` | Explain like I'm 5 | `cat quantum.txt \| eli5` |
| `tldr` | One-line summary | `cat readme.txt \| tldr` |
| `expand` | Expand bullet points | `cat notes.txt \| expand` |

### Developer Tools

| Tool | Description | Example |
|------|-------------|---------|
| `explain-error` | Explain stack traces | `cat error.log \| explain-error` |
| `explain-code` | Explain what code does | `cat script.py \| explain-code` |
| `review-code` | Quick code review | `cat pr.diff \| review-code --focus security` |
| `gen-tests` | Generate unit tests | `cat module.py \| gen-tests --framework pytest` |
| `docstring` | Add docstrings | `cat functions.py \| docstring` |
| `commit-msg` | Generate commit message | `git diff --staged \| commit-msg` |

### Data Tools

| Tool | Description | Example |
|------|-------------|---------|
| `json-extract` | Extract as validated JSON | `cat text.txt \| json-extract --fields "name, email"` |
| `json2csv` | Convert JSON to CSV | `cat data.json \| json2csv` |
| `extract-emails` | Extract email addresses | `cat page.html \| extract-emails` |
| `extract-contacts` | Extract contacts as CSV | `cat notes.txt \| extract-contacts` |
| `sql-from-text` | Natural language to SQL | `echo "get active users" \| sql-from-text` |
| `safe-sql` | SQL with safety checks | `echo "delete old records" \| safe-sql` |
| `parse-log` | Analyze log files | `cat app.log \| parse-log --focus errors` |
| `csv-insights` | Analyze CSV data | `cat sales.csv \| csv-insights --question "trends?"` |

### Advanced Multi-Step Tools

| Tool | Description | Pattern |
|------|-------------|---------|
| `log-errors` | Extract & explain errors from huge logs | Code→AI |
| `diff-focus` | Review only added lines | Code→AI |
| `changelog` | Git log to formatted changelog | Code→AI→Code |
| `code-validate` | Generate syntax-checked Python | AI→Code |

### Install Example Tools

```bash
# Run the example installer (from the CmdForge directory)
python examples/install.py
cmdforge refresh
```

## Providers

CmdForge works with any AI CLI tool. We've profiled 12 providers:

| Provider | Speed | Accuracy | Cost | Best For |
|----------|-------|----------|------|----------|
| `opencode-deepseek` | 13s | 4/4 | Cheap | **Best value** - daily use |
| `opencode-pickle` | 13s | 4/4 | FREE | **Best free** - accurate |
| `claude-haiku` | 14s | 4/4 | Paid | Fast + accurate |
| `codex` | 14s | 4/4 | Paid | Reliable |
| `claude-opus` | 18s | 4/4 | $$$ | Highest quality |
| `gemini` | 91s | 3/4 | Paid | Large docs (1M tokens) |

See [Provider Setup](docs/reference/providers.md) for setup instructions.

## Tool Anatomy

A tool is a YAML config file in `~/.cmdforge/<name>/config.yaml`:

```yaml
name: summarize
description: Condense documents to key points
category: Text  # Optional: Text, Developer, Data, or Other
arguments:
  - flag: --length
    variable: length
    default: "3-5 bullet points"
steps:
  - type: prompt
    prompt: |
      Summarize this text into {length}:

      {input}      
    provider: opencode-pickle
    output_var: response
output: "{response}"
```

### Categories

Tools can be organized into categories for easier navigation in the UI:

| Category | Description |
|----------|-------------|
| `Text` | Text processing (summarize, translate, grammar) |
| `Developer` | Dev tools (explain-error, gen-tests, commit-msg) |
| `Data` | Data extraction and conversion (json-extract, csv) |
| `Other` | Default for uncategorized tools |

### Multi-Step Tools

Chain AI prompts with Python code for powerful workflows:

```yaml
name: json-extract
description: Extract data as validated JSON
steps:
  # Step 1: AI extracts data
  - type: prompt
    prompt: "Extract {fields} as JSON from: {input}"
    provider: opencode-deepseek
    output_var: raw_json

  # Step 2: Code validates JSON
  - type: code
    code: |
      import json
      try:
          parsed = json.loads(raw_json)
          validated = json.dumps(parsed, indent=2)
      except:
          validated = "ERROR: Invalid JSON"      
    output_var: validated
output: "{validated}"
```

### Meta-Tools (Tools Calling Tools)

Tools can call other tools as steps, enabling powerful composition:

```yaml
name: summarize-and-translate
description: Summarize text then translate the summary
dependencies:
  - official/summarize
  - official/translate
steps:
  - type: tool
    tool: official/summarize
    input: "{input}"
    args:
      max_words: "100"
    output_var: summary

  - type: tool
    tool: official/translate
    input: "{summary}"
    args:
      target_language: "{language}"
    output_var: translated
output: "{translated}"
```

Check dependencies before running: `cmdforge check my-meta-tool`

See [Meta-Tools Design](docs/reference/meta-tools.md) for full documentation.

### Variables

| Variable | Description |
|----------|-------------|
| `{input}` | The piped/file input |
| `{argname}` | Custom argument value |
| `{output_var}` | Output from previous step (prompt, code, or tool) |

## Project Dependencies

For projects that use multiple CmdForge tools, you can declare dependencies in a `cmdforge.yaml` manifest:

```yaml
# cmdforge.yaml
name: my-project
description: My awesome project
dependencies:
  - official/summarize
  - official/translate
  - myuser/custom-tool@^1.0.0
```

### Managing Dependencies

```bash
# Initialize a new manifest
cmdforge init

# Add a dependency
cmdforge add official/summarize
cmdforge add myuser/tool@^2.0.0    # With version constraint

# Install all dependencies
cmdforge install

# Check dependency status
cmdforge deps
```

This ensures all required tools are available before running your project's scripts.

## Collections

Collections are curated bundles of related tools. Install an entire workflow with one command.

### Registry Collections

Browse and install collections from the CmdForge registry:

```bash
# List available collections from registry
cmdforge collections list

# View what's in a collection
cmdforge collections info starter
# Output:
#   starter - Essential tools for getting started
#   Tools: summarize, translate, fix-grammar, explain-error, commit-msg
#   Total: 5 tools

# Install all tools in a collection
cmdforge collections install starter
```

### Local Collections

Create and manage your own collections locally:

```bash
# List your local collections
cmdforge collections list --local

# Create a new local collection
cmdforge collections create my-toolkit

# View collection details
cmdforge collections show my-toolkit

# Add tools to the collection
cmdforge collections add my-toolkit summarize
cmdforge collections add my-toolkit official/translate

# Remove tools from the collection
cmdforge collections remove my-toolkit summarize

# Delete a collection
cmdforge collections delete my-toolkit
```

### Publishing Collections

Share your collections with the community:

```bash
# Check collection status before publishing
cmdforge collections status my-toolkit
# Shows: tool visibility, approval status, any issues

# Publish to registry (requires account)
cmdforge collections publish my-toolkit

# Dry run to see what would be published
cmdforge collections publish my-toolkit --dry-run

# Force publish even with warnings
cmdforge collections publish my-toolkit --force

# Continue publishing after fixing issues
cmdforge collections publish my-toolkit --continue
```

**Publishing requirements:**
- All tools must be public (not private visibility)
- All tools must have an approved version in the registry
- Local tools will be prefixed with your username (e.g., `summarize` → `yourname/summarize`)

### Available Registry Collections

| Collection | Description | Tools |
|------------|-------------|-------|
| `starter` | Essential tools for beginners | 5 |
| `developer` | Code review and development | 8 |
| `writer` | Writing and editing tools | 6 |
| `data` | Data extraction and conversion | 7 |

## Registry

The CmdForge Registry at [cmdforge.brrd.tech](https://cmdforge.brrd.tech) hosts community tools.

### Searching Tools

```bash
# Basic search
cmdforge registry search "summarize"

# Filter by category
cmdforge registry search --category Developer

# Sort by popularity
cmdforge registry search --sort downloads

# Show only new tools (last 7 days)
cmdforge registry search --new

# Advanced filters
cmdforge registry search --owner official --min-downloads 100
```

### Publishing Tools

```bash
# Connect your account (opens browser for authentication)
cmdforge config connect yourusername

# Publish a tool
cmdforge registry publish mytool

# Check moderation status
cmdforge registry status
# Output:
#   mytool: pending (submitted 2 hours ago)
#   other-tool: approved

# Sync moderation feedback to your local tool
cmdforge registry status --sync
```

Published tools go through moderation before appearing publicly. You'll receive feedback if changes are requested.

### Tool Documentation

Each tool can have its own README:

```bash
# View tool documentation
cmdforge docs mytool

# Create/edit documentation (opens in $EDITOR)
cmdforge docs mytool --edit
```

The documentation is stored as `README.md` in the tool's directory (`~/.cmdforge/mytool/README.md`).

## Shell Integration

### Git Hooks

```bash
# .git/hooks/prepare-commit-msg
#!/bin/bash
git diff --cached | commit-msg > "$1"
```

### Aliases

```bash
# ~/.bashrc
alias gc='git diff --staged | commit-msg'
alias wtf='explain-error'
alias fixme='fix-grammar'
```

### Vim Integration

```vim
" Fix grammar in selection
vnoremap <leader>fg :!fix-grammar<CR>

" Explain selected code
vnoremap <leader>ec :!explain-code<CR>
```

## GUI Features

The graphical interface provides a modern desktop experience:

### My Tools Page
- View all your tools organized by category (Text, Developer, Data, Other)
- Double-click a tool to edit it
- Create new tools with the built-in Tool Builder
- Connect to the registry to publish your tools

### Collections Page
- **Local Collections**: Create and manage your own tool bundles
  - Create new collections with display name and description
  - Add/remove tools from collections
  - View collection details and tool lists
- **Registry Collections**: Browse collections from the registry
  - Search and filter available collections
  - View collection contents before installing
  - One-click install for all tools in a collection
- **Publish Collections**: Share your collections with the community
  - Automatic tool resolution (local names → registry references)
  - Visibility and approval status checks
  - Options to publish tools first, skip unpublished, or cancel

### Tool Builder
- Visual form for creating and editing tools
- Add arguments with flags and default values
- Add prompt steps (AI calls) with profile selection
- Add code steps with **AI-assisted code generation**:
  - Split view: code editor + AI assist panel
  - Describe what you want and generate Python code
  - Shows available variables from arguments and previous steps
  - Automatic markdown fence stripping from AI responses

### Registry Browser
- **Browse all** tools on page load (no search required)
- **Search** by name, description, or keyword
- **Filter** by category (Text, Developer, Data, Other)
- **Sort** by popularity, rating, newest, or name
- **Star ratings** displayed in results and details
- **Clickable tags** to filter by tag
- **Installed indicator** (✓) shows which tools you have
- **Update available** (↑) when newer version exists
- **Pagination** for browsing large result sets
- **Publisher info** showing reputation and total downloads

### Provider Management
- Add and configure AI providers
- Test provider connectivity
- Set default providers for new tools

### Profiles (AI Personas)
- **Built-in profiles**: Comedian, Technical Writer, Teacher, Concise, Creative, Code Reviewer, Analyst
- **Custom profiles**: Create your own personas with custom system prompts
- **Profile selector**: Choose a profile when adding prompt steps to tools
- **Automatic injection**: Profile system prompts are prepended to your prompts during execution

### Keyboard Shortcuts

| Shortcut | Action |
|----------|--------|
| `Ctrl+N` | Create new tool |
| `Ctrl+S` | Save tool (in builder) |
| `Ctrl+R` | Refresh current page |
| `Ctrl+1` | Go to My Tools |
| `Ctrl+2` | Go to Registry |
| `Ctrl+3` | Go to Collections |
| `Ctrl+4` | Go to Providers |
| `Ctrl+5` | Go to Profiles |
| `Escape` | Close tool builder |
| `Ctrl+Q` | Quit application |

## Multi-Step Tool Patterns

CmdForge supports powerful multi-step workflows combining AI and code:

### Code → AI
Extract or transform data with Python, then pass to AI:
```yaml
steps:
  - type: code
    code: |
      # Extract just the error lines
      errors = [l for l in input_text.split('\n') if 'error' in l.lower()]
      result = '\n'.join(errors[-10:])  # Last 10 errors      
    output_var: errors
  - type: prompt
    prompt: "Explain these errors:\n{errors}"
    output_var: explanation
```

### AI → Code
Generate content with AI, then validate/process with Python:
```yaml
steps:
  - type: prompt
    prompt: "Generate Python code to: {input}"
    output_var: generated
  - type: code
    code: |
      import ast
      try:
          ast.parse(generated)  # Syntax check
          result = generated
      except SyntaxError as e:
          result = f"# Syntax Error: {e}\n{generated}"      
    output_var: validated
```

## Philosophy

CmdForge is built on **Unix philosophy**:

1. **Do one thing well** - Each tool solves one problem. `summarize` summarizes. `translate` translates. No bloated mega-tools.

2. **Compose freely** - Tools read stdin, write stdout. Chain them: `cat doc.txt | summarize | translate --lang French`

3. **Text is the interface** - No proprietary formats. Pipe text in, get text out. Works with `grep`, `awk`, `sed`, and every other Unix tool.

4. **You own everything** - Tools are YAML files. Prompts are plain text. No vendor lock-in, no cloud dependency for your tool definitions.

5. **Swap parts freely** - Change AI providers per-tool or per-run. Today's best model might not be tomorrow's.

## Contributing

```bash
git clone https://gitea.brrd.tech/rob/cmdforge.git
cd cmdforge
pip install -e ".[dev]"
pytest
```

## Support

If CmdForge saves you time, consider supporting the project:

**Bitcoin:** `3KeLqGv2Xo8jGYkqA1i68a6nXwgQnuJoza`

**Questions?** robdickson444@hotmail.com

## License

MIT - Use it, modify it, share it.

## Docker

### Pre-installed Container (Recommended)

Ready-to-use container with CmdForge and 27 example tools pre-installed:

```bash
# Build from source
docker build -f Dockerfile.ready -t cmdforge-ready .

# Run interactively
docker run -it --rm cmdforge-ready

# With persistent storage (keeps your tools between runs)
docker run -it --rm -v cmdforge-data:/home/user/.cmdforge cmdforge-ready
```

Inside the container, CmdForge is ready to use:
```bash
cmdforge list            # See 27 pre-installed tools
cmdforge                 # Launch the GUI (requires display)
cmdforge run summarize   # Run a tool
```

### Test Installation Container

Simulates a fresh computer for testing the installation process:

```bash
# Build the test container
docker build -f Dockerfile.test-install -t cmdforge-test-install .

# Run interactively
docker run -it --rm cmdforge-test-install

# Inside the container - run the installer
cd ~/CmdForge
./install.sh
```

### Interactive Installer

The `install.sh` script provides guided installation:

```bash
./install.sh              # Full interactive install
./install.sh --yes        # Non-interactive (accept defaults)
./install.sh --no-venv    # Install without virtual environment
./install.sh --no-examples # Skip installing example tools
```

Features:
- Creates virtual environment at `~/.cmdforge-venv`
- Configures PATH in your shell config (bash/zsh/fish)
- Optionally installs 27 example tools
- Works on macOS and Linux

### Legacy Container

The original docker-compose setup still works:

```bash
# Build the container
docker-compose build

# Run tests
docker-compose run --rm test

# Use the CLI
docker-compose run --rm cli cmdforge list

# Interactive shell
docker-compose run --rm shell
```

### Installing AI Providers in Docker

AI providers require browser-based authentication. You need display access for the browser to open.

**Using the pre-built container:**

```bash
# Allow Docker to access your display (Linux)
xhost +local:docker

# Run with display access and persistent storage
docker run -it --rm \
  -e DISPLAY=$DISPLAY \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -v cmdforge-data:/root/.cmdforge \
  --network host \
  gitea.brrd.tech/rob/cmdforge bash

# Inside the container:
cmdforge providers install
```

**Using docker-compose (build from source):**

```bash
xhost +local:docker
docker-compose run --rm setup

# Inside the container:
cmdforge providers install
```

**After installing a provider:**

```bash
# Test the provider works
cmdforge providers test claude
```

**Available Providers:**
| # | Provider | Cost | Notes |
|---|----------|------|-------|
| 1 | Claude | Pay-per-use | Anthropic account |
| 2 | Codex | Pay-per-use | OpenAI account |
| 3 | Gemini | Free tier | Google account |
| 4 | OpenCode | Free tier | Multiple models |
| 5 | Ollama | FREE | Local, no internet |

## Links

### User Documentation
- [Provider Setup](docs/reference/providers.md)
- [Example Tools](docs/reference/examples.md)
- [User Wiki](wiki/Home.md)

### Developer Documentation
- [Project Overview](docs/overview.md) - Start here to understand the codebase
- [Design Document](docs/reference/design.md)
- [Meta-Tools Design](docs/reference/meta-tools.md) - Tools that call other tools
- [Collections](docs/reference/collections.md) - Curated tool bundles
- [Registry API Design](docs/reference/registry-spec.md)
- [Web UI Design](docs/reference/web-ui-spec.md)