Add comprehensive documentation and 28 example tools

- Updated README with full feature overview and examples
- Added docs/INSTALL.md with detailed setup instructions
- Added docs/PROVIDERS.md with 12 profiled providers
- Added docs/EXAMPLES.md with all 28 tool configurations
- Added examples/install.py for one-command tool installation
- Updated pyproject.toml with better metadata
- Added urwid TUI with provider descriptions in dropdown
- Profiled all providers for speed/accuracy/cost

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
This commit is contained in:
rob
2025-12-04 12:42:04 -04:00
co-authored by Claude
parent 28dac465e6
commit 4c71dbded2
14 changed files with 6072 additions and 565 deletions
+215 -155
View File
@@ -1,10 +1,10 @@
# SmartTools - Simple Design
# SmartTools - Design Document
> A lightweight personal tool builder for AI-powered CLI commands
## Overview
SmartTools lets you create custom AI-powered terminal commands. You define a tool once (name, prompt, provider), then use it like any Linux command.
SmartTools lets you create custom AI-powered terminal commands. You define a tool once (name, steps, provider), then use it like any Linux command.
**Example:**
```bash
@@ -20,44 +20,91 @@ sum -i text.txt -o summary.txt --max 512
~/.smarttools/
sum/
config.yaml
post.py # optional post-processing script
processed.py # Optional external code file
reviewer/
config.yaml
translator/
config.yaml
```
### Minimal config.yaml
### config.yaml Format
```yaml
name: sum
description: "Summarize documents"
prompt: |
Summarize the following text in {max} words or less:
arguments:
- flag: --max
variable: max
default: "500"
description: "Maximum words in summary"
steps:
- type: prompt
prompt: |
Summarize the following text in {max} words or less:
{input}
provider: codex
provider_args: "-p"
# Optional
inputs:
- name: max
flag: --max
default: 500
# Optional post-processing script
post_process: post.py
{input}
provider: claude
output_var: response
output: "{response}"
```
That's it. No trust tiers, no signing, no containers.
### Step Types
**Prompt Step** - Calls an AI provider:
```yaml
- type: prompt
prompt: "Your prompt template with {variables}"
provider: claude
output_var: response
```
**Code Step** - Runs Python code:
```yaml
- type: code
code: |
processed = input.upper()
count = len(processed.split())
output_var: processed, count
code_file: processed.py # Optional: external file storage
```
Steps execute in order. Each step's `output_var` becomes available to subsequent steps.
### Variables
- `{input}` - Always available, contains stdin or input file content (empty string if no input)
- `{variable_name}` - From arguments (e.g., `{max}`)
- `{output_var}` - From previous steps (e.g., `{response}`, `{processed}`)
### Output Variables
The `output_var` field specifies which Python variable(s) to capture from your code:
**Single variable:**
```yaml
output_var: processed
```
```python
processed = input.upper() # This gets captured
```
**Multiple variables (comma-separated):**
```yaml
output_var: processed, count, summary
```
```python
processed = input.upper()
count = len(processed.split())
summary = f"Processed {count} words"
# All three are captured and available as {processed}, {count}, {summary}
```
## CLI Interface
### Running Tools
```bash
# Basic usage
# Basic usage (wrapper script in ~/.local/bin)
sum -i document.txt -o summary.txt
# With custom args
@@ -74,14 +121,33 @@ sum -i document.txt --provider mock
# Read from stdin, write to stdout
cat doc.txt | sum | less
# Interactive stdin input
sum --stdin
# No input (empty string) - useful for tools using only arguments
sum --max 100
# Or via smarttools run
smarttools run sum -i document.txt
```
### Input Handling
| Scenario | Behavior |
|----------|----------|
| Piped stdin | Automatically read (`cat file.txt \| mytool`) |
| `-i file.txt` | Read from file |
| `--stdin` | Interactive input (type then Ctrl+D) |
| No input | Empty string (useful for argument-only tools) |
### Universal Flags (all tools)
| Flag | Short | Description |
|------|-------|-------------|
| `--input` | `-i` | Input file (or stdin if omitted) |
| `--input` | `-i` | Input file |
| `--output` | `-o` | Output file (or stdout if omitted) |
| `--stdin` | | Read input interactively (type then Ctrl+D) |
| `--dry-run` | | Show prompt, don't call AI |
| `--show-prompt` | | Call AI but also print prompt to stderr |
| `--provider` | `-p` | Override provider (e.g., `--provider mock`) |
@@ -96,15 +162,18 @@ smarttools
# Or use CLI directly:
smarttools list # List all tools
smarttools create sum # Create new tool
smarttools edit sum # Edit existing tool
smarttools create sum # Create new tool (basic)
smarttools edit sum # Edit tool config in $EDITOR
smarttools delete sum # Delete tool
smarttools test sum # Test with mock provider
smarttools run sum # Run tool for real
smarttools refresh # Refresh all wrapper scripts
smarttools ui # Launch interactive UI
```
## Lightweight UI
## Terminal UI
A simple terminal UI using `dialog` or `whiptail` (available on most Linux systems).
A BIOS-style terminal UI using `urwid` with full mouse support.
### Main Menu
@@ -112,66 +181,84 @@ A simple terminal UI using `dialog` or `whiptail` (available on most Linux syste
┌──────────────────────────────────────┐
│ SmartTools Manager │
├──────────────────────────────────────┤
│ > List Tools │
│ Create New Tool │
│ Edit Tool │
│ Delete Tool │
│ Test Tool │
│ Exit │
│ Tools: │
│ ┌────────────────────────────────┐ │
│ │ sum │ │
│ │ reviewer │ │
│ │ translator │ │
│ └────────────────────────────────┘ │
│ │
│ [Create] [Edit] [Delete] [Test] │
│ │
│ Tool Info: │
│ Name: sum │
│ Description: Summarize documents │
│ Arguments: --max │
│ Steps: PROMPT[claude] -> {response} │
│ │
│ [EXIT] │
└──────────────────────────────────────┘
```
### Create/Edit Tool Form
### Tool Builder
```
┌──────────────────────────────────────┐
│ Create New Tool │
├──────────────────────────────────────┤
│ Name: [sum________________] │
│ Description: [Summarize documents_] │
│ Provider: [codex_____________] │
│ Provider Args: [-p________________] │
│ │
│ Prompt: │
│ ┌──────────────────────────────────┐ │
│ │ Summarize the following text │ │
│ │ in {max} words or less: │ │
│ │ │ │
│ │ {input} │ │
│ └──────────────────────────────────┘ │
│ │
│ Custom Arguments: │
│ --max (default: 500) │
│ [Add Argument] │
│ │
│ Post-process Script: [none______▼] │
│ │
│ [Save] [Test] [Cancel] │
└──────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────┐
│ New Tool │
├──────────────────────────────────────────────────────────────┤
│ ┌─[ Tool Info ]──────────┐ ┌─ Arguments ─────────────────┐ │
│ │ Name: [sum__________] │ │ --max -> {max} │ │
│ │ │ │ │ │
│ │ Desc: [Summarize____] │ └─────────────────────────────┘ │
│ │ │ [Add] [Edit] [Delete] │
│ │ Output: [{response}_] │ │
│ └────────────────────────┘ ┌─ Execution Steps ───────────┐ │
│ │ P:claude -> {response} │ │
│ │ │ │
│ └─────────────────────────────┘ │
│ [Add] [Edit] [Delete] │
│ │
│ [Save] [Cancel] │
└──────────────────────────────────────────────────────────────┘
```
### Test Tool
**Navigation:**
- **Tab** - Cycle between sections (Tool Info -> Arguments -> Steps -> Buttons)
- **Arrow keys** - Navigate within lists
- **Enter/Click** - Select/activate
- **Escape** - Cancel/back
- **Shift + mouse** - Terminal-native text selection (bypasses UI mouse handling)
- **Ctrl+Shift+V** - Paste
The current section's title is highlighted with brackets: `[ Tool Info ]`
**Copy/Paste tip:** Hold `Shift` while dragging with mouse to select text, then use your terminal's copy function (usually right-click or Ctrl+Shift+C).
### Code Step Dialog
```
┌──────────────────────────────────────┐
│ Test Tool: sum │
├──────────────────────────────────────┤
│ Input: [Select file...] or [Paste] │
│ │
│ ┌─ Final Prompt ───────────────────┐ │
│ │ Summarize the following text │ │
│ │ in 500 words or less: │ │
│ │ │ │
│ │ Lorem ipsum dolor sit amet... │ │
│ └──────────────────────────────────┘ │
│ │
│ Provider: [mock_____▼] │
│ │
│ [Run Test] [Copy Command] │
└──────────────────────────────────────┘
┌──────────────── Add Code Step ─────────────────┐
│ Variables: input, max, response │
│ │
│ File: [processed.py______________] < Load > │
│ │
│ ┌─ Code ─────────────────────────────────────┐ │
│ │processed = input.upper() │ │
│ │count = len(processed.split()) │ │
│ │ │ │
│ └────────────────────────────────────────────┘ │
│ │
│ Output var: [processed, count____] │
├────────────────────────────────────────────────┤
│ < OK > < Cancel > │
└────────────────────────────────────────────────┘
```
"Copy Command" shows: `sum -i test.txt --dry-run`
**Code Step Features:**
- **Multiline editor** - Write multi-line Python code
- **External file storage** - Code is auto-saved to `~/.smarttools/<toolname>/<filename>` on OK
- **Load button** - Load code from external file (with confirmation)
- **Multiple output vars** - Capture multiple variables (comma-separated)
## Implementation
@@ -181,79 +268,54 @@ A simple terminal UI using `dialog` or `whiptail` (available on most Linux syste
smarttools/
__init__.py
cli.py # Entry point, argument parsing
ui.py # Dialog-based UI
tool.py # Tool loading/saving
ui.py # UI dispatcher (chooses urwid/snack/dialog)
ui_urwid.py # Urwid-based BIOS-style UI
ui_snack.py # Snack/newt fallback UI
tool.py # Tool loading/saving/wrapper scripts
runner.py # Execute tools
providers.py # Provider abstraction (minimal)
providers.py # Provider abstraction
```
### Provider Abstraction (Simple)
### Provider System
```python
# providers.py
import subprocess
Providers are defined in `~/.smarttools/providers.yaml`:
def call_provider(provider: str, args: str, prompt: str) -> str:
"""Call an AI CLI tool with the given prompt."""
cmd = f"{provider} {args}"
result = subprocess.run(
cmd,
shell=True,
input=prompt,
capture_output=True,
text=True
)
if result.returncode != 0:
raise RuntimeError(f"Provider failed: {result.stderr}")
return result.stdout
def mock_provider(prompt: str) -> str:
"""Return a mock response for testing."""
return f"[MOCK RESPONSE]\nReceived prompt of {len(prompt)} chars"
```yaml
providers:
- name: claude
command: "claude -p"
description: "Anthropic Claude"
- name: codex
command: "codex -p"
description: "OpenAI Codex"
- name: gemini
command: "gemini"
description: "Google Gemini"
```
### Tool Runner (Simple)
The provider command receives the prompt via stdin and outputs to stdout.
```python
# runner.py
import yaml
from pathlib import Path
### Wrapper Scripts
TOOLS_DIR = Path.home() / ".smarttools"
def load_tool(name: str) -> dict:
config_path = TOOLS_DIR / name / "config.yaml"
return yaml.safe_load(config_path.read_text())
def build_prompt(tool: dict, input_text: str, args: dict) -> str:
prompt = tool["prompt"]
prompt = prompt.replace("{input}", input_text)
for key, value in args.items():
prompt = prompt.replace(f"{{{key}}}", str(value))
return prompt
def run_tool(name: str, input_text: str, args: dict, provider_override: str = None) -> str:
tool = load_tool(name)
prompt = build_prompt(tool, input_text, args)
provider = provider_override or tool["provider"]
if provider == "mock":
return mock_provider(prompt)
return call_provider(provider, tool.get("provider_args", ""), prompt)
```
### Generated Wrapper Script
When you create a tool, SmartTools generates a wrapper script:
When you save a tool, SmartTools creates a wrapper script in `~/.local/bin/`:
```bash
#!/bin/bash
# ~/.smarttools/sum/sum (auto-generated)
exec python3 -m smarttools.run sum "$@"
# SmartTools wrapper for 'sum'
# Auto-generated - do not edit
exec /path/to/python -m smarttools.runner sum "$@"
```
And symlinks it to `~/.local/bin/sum` so you can call `sum` directly.
The wrapper uses the full Python path to ensure the correct environment is used.
Use `smarttools refresh` to regenerate all wrapper scripts (e.g., after changing Python environments).
### Tool Name Validation
Tool names must:
- Start with a letter or underscore
- Contain only letters, numbers, underscores, and dashes
- Not contain spaces or shell-problematic characters (`/\|&;$`"'<>(){}[]!?*#~`)
## What This Design Doesn't Include
@@ -272,33 +334,31 @@ Intentionally omitted (not needed for personal use):
## Dependencies
Minimal:
Required:
- Python 3.10+
- PyYAML
- `dialog` or `whiptail` (pre-installed on most Linux)
- urwid (for TUI)
Optional:
- `textual` (if you want a fancier TUI later)
## File Sizes
Estimated implementation:
- `cli.py`: ~100 lines
- `ui.py`: ~150 lines
- `tool.py`: ~80 lines
- `runner.py`: ~60 lines
- `providers.py`: ~40 lines
**Total: ~430 lines of Python**
Optional fallbacks:
- python3-newt/snack (simpler TUI)
- dialog/whiptail (basic TUI)
## Example Workflow
1. Run `smarttools` to open UI
2. Select "Create New Tool"
3. Fill in: name=`sum`, prompt, provider=`codex`
4. Click "Test" to verify with mock provider
5. Click "Save"
6. Exit UI
7. Run `sum -i myfile.txt -o summary.txt`
2. Select "Create" to create a new tool
3. Fill in: name, description, output template
4. Add arguments (e.g., `--max` with default `500`)
5. Add a prompt step with your prompt template and provider
6. Click "Save"
7. Exit UI
8. Run `sum -i myfile.txt -o summary.txt`
Done. No containers, no signing, no certification. Just a tool that works.
## Default Values
| Step Type | Default Output Var | Default File |
|-----------|-------------------|--------------|
| Prompt | `response` | n/a |
| Code | `processed` | `processed.py` |
+960
View File
@@ -0,0 +1,960 @@
# Example Tools
This document contains all 28 pre-built SmartTools with their full configurations.
## Quick Install
```bash
# Install all example tools
curl -sSL https://raw.githubusercontent.com/yourusername/smarttools/main/examples/install.py | python3
smarttools refresh
```
## Text Processing Tools
### summarize
Condense long documents to key points.
```yaml
# ~/.smarttools/summarize/config.yaml
name: summarize
description: Condense long documents to key points
arguments:
- flag: --length
variable: length
default: "3-5 bullet points"
steps:
- type: prompt
prompt: |
Summarize the following text into {length}. Be concise and capture the key points:
{input}
provider: opencode-pickle
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat article.txt | summarize
cat book.txt | summarize --length "10 bullet points"
```
---
### translate
Translate text to any language.
```yaml
name: translate
description: Translate text to any language
arguments:
- flag: --lang
variable: lang
default: Spanish
steps:
- type: prompt
prompt: |
Translate the following text to {lang}. Only output the translation, nothing else:
{input}
provider: claude-haiku
output_var: response
output: "{response}"
```
**Usage:**
```bash
echo "Hello, world!" | translate --lang French
cat readme.md | translate --lang Japanese
```
---
### fix-grammar
Fix grammar and spelling errors.
```yaml
name: fix-grammar
description: Fix grammar and spelling errors
arguments: []
steps:
- type: prompt
prompt: |
Fix all grammar, spelling, and punctuation errors in the following text. Only output the corrected text, no explanations:
{input}
provider: opencode-deepseek
output_var: response
output: "{response}"
```
**Usage:**
```bash
echo "teh cat sat on teh mat" | fix-grammar
cat draft.txt | fix-grammar > fixed.txt
```
---
### simplify
Rewrite text for easier understanding.
```yaml
name: simplify
description: Rewrite text for easier understanding
arguments:
- flag: --level
variable: level
default: "5th grade reading level"
steps:
- type: prompt
prompt: |
Rewrite the following text for a {level}. Keep the meaning but use simpler words and shorter sentences:
{input}
provider: opencode-pickle
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat legal_document.txt | simplify
cat technical.md | simplify --level "non-technical reader"
```
---
### tone-shift
Change the tone of text.
```yaml
name: tone-shift
description: Change the tone of text
arguments:
- flag: --tone
variable: tone
default: professional
steps:
- type: prompt
prompt: |
Rewrite the following text in a {tone} tone. Keep the core message but adjust the style:
{input}
provider: opencode-deepseek
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat angry_email.txt | tone-shift --tone "calm and professional"
cat casual_note.txt | tone-shift --tone formal
```
---
### eli5
Explain like I'm 5.
```yaml
name: eli5
description: Explain like I'm 5
arguments: []
steps:
- type: prompt
prompt: |
Explain this like I'm 5 years old. Use simple words and fun analogies:
{input}
provider: opencode-pickle
output_var: response
output: "{response}"
```
**Usage:**
```bash
echo "What is quantum computing?" | eli5
cat whitepaper.txt | eli5
```
---
### tldr
One-line summary.
```yaml
name: tldr
description: One-line summary
arguments: []
steps:
- type: prompt
prompt: |
Give a one-line TL;DR summary of this text:
{input}
provider: opencode-grok
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat long_article.txt | tldr
curl -s https://example.com | tldr
```
---
### expand
Expand bullet points to paragraphs.
```yaml
name: expand
description: Expand bullet points to paragraphs
arguments: []
steps:
- type: prompt
prompt: |
Expand these bullet points into well-written paragraphs:
{input}
provider: opencode-pickle
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat notes.txt | expand
echo "- Fast\n- Reliable\n- Easy to use" | expand
```
---
## Developer Tools
### explain-error
Explain error messages and stack traces.
```yaml
name: explain-error
description: Explain error messages and stack traces
arguments: []
steps:
- type: prompt
prompt: |
Explain this error/stack trace in plain English. What went wrong and how to fix it:
{input}
provider: claude-haiku
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat error.log | explain-error
python script.py 2>&1 | explain-error
```
---
### explain-code
Explain what code does.
```yaml
name: explain-code
description: Explain what code does
arguments:
- flag: --detail
variable: detail
default: moderate
steps:
- type: prompt
prompt: |
Explain what this code does at a {detail} level of detail:
```
{input}
```
provider: opencode-pickle
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat script.py | explain-code
cat complex.js | explain-code --detail "very detailed"
```
---
### review-code
Quick code review with suggestions.
```yaml
name: review-code
description: Quick code review with suggestions
arguments:
- flag: --focus
variable: focus
default: "bugs, security, and improvements"
steps:
- type: prompt
prompt: |
Review this code focusing on {focus}. Be concise and actionable:
```
{input}
```
provider: claude-sonnet
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat pull_request.diff | review-code
cat auth.py | review-code --focus "security vulnerabilities"
```
---
### gen-tests
Generate unit tests for code.
```yaml
name: gen-tests
description: Generate unit tests for code
arguments:
- flag: --framework
variable: framework
default: pytest
steps:
- type: prompt
prompt: |
Generate comprehensive unit tests for this code using {framework}. Include edge cases:
```
{input}
```
provider: claude-haiku
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat utils.py | gen-tests
cat api.js | gen-tests --framework jest
```
---
### docstring
Add docstrings to functions/classes.
```yaml
name: docstring
description: Add docstrings to functions/classes
arguments:
- flag: --style
variable: style
default: Google style
steps:
- type: prompt
prompt: |
Add {style} docstrings to all functions and classes in this code. Output the complete code with docstrings:
```
{input}
```
provider: opencode-deepseek
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat module.py | docstring
cat functions.py | docstring --style "NumPy style"
```
---
### commit-msg
Generate commit message from diff.
```yaml
name: commit-msg
description: Generate commit message from diff
arguments:
- flag: --style
variable: style
default: conventional commits
steps:
- type: prompt
prompt: |
Generate a concise {style} commit message for this diff. Just the message, no explanation:
{input}
provider: opencode-pickle
output_var: response
output: "{response}"
```
**Usage:**
```bash
git diff --staged | commit-msg
git diff HEAD~1 | commit-msg --style "simple"
```
---
## Data Tools
### json-extract
Extract structured data as validated JSON.
```yaml
name: json-extract
description: Extract structured data as validated JSON
arguments:
- flag: --fields
variable: fields
default: any relevant fields
steps:
- type: prompt
prompt: |
Extract {fields} from this text as a JSON object. Output ONLY valid JSON, no markdown, no explanation:
{input}
provider: opencode-deepseek
output_var: raw_json
- type: code
code: |
import json
import re
text = raw_json.strip()
text = re.sub(r'^```json?\s*', '', text)
text = re.sub(r'\s*```$', '', text)
try:
parsed = json.loads(text)
validated = json.dumps(parsed, indent=2)
except json.JSONDecodeError as e:
validated = f"ERROR: Invalid JSON - {e}\nRaw output: {text[:500]}"
output_var: validated
output: "{validated}"
```
**Usage:**
```bash
echo "Price $49.99, SKU ABC-123" | json-extract --fields "price, sku"
cat invoice.txt | json-extract --fields "total, date, items"
```
---
### json2csv
Convert JSON to CSV format.
```yaml
name: json2csv
description: Convert JSON to CSV format
arguments: []
steps:
- type: prompt
prompt: |
Convert this JSON to CSV format. Output only the CSV, no explanation:
{input}
provider: opencode-deepseek
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat data.json | json2csv
echo '[{"name":"John","age":30}]' | json2csv
```
---
### extract-emails
Extract email addresses from text.
```yaml
name: extract-emails
description: Extract email addresses from text
arguments: []
steps:
- type: prompt
prompt: |
Extract all email addresses from this text. Output one email per line, nothing else:
{input}
provider: opencode-pickle
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat webpage.html | extract-emails
cat contacts.txt | extract-emails | sort -u
```
---
### extract-contacts
Extract contact info as structured CSV.
```yaml
name: extract-contacts
description: Extract contact info as structured CSV
arguments: []
steps:
- type: prompt
prompt: |
Extract all contact information (name, email, phone, company) from this text. Output as JSON array with objects having keys: name, email, phone, company. Use null for missing fields. Output ONLY the JSON array:
{input}
provider: opencode-pickle
output_var: contacts_json
- type: code
code: |
import json
import re
text = contacts_json.strip()
text = re.sub(r'^```json?\s*', '', text)
text = re.sub(r'\s*```$', '', text)
try:
contacts = json.loads(text)
lines = ['name,email,phone,company']
for c in contacts:
row = [
str(c.get('name') or ''),
str(c.get('email') or ''),
str(c.get('phone') or ''),
str(c.get('company') or '')
]
lines.append(','.join(f'"{f}"' for f in row))
csv_output = '\n'.join(lines)
except:
csv_output = f"Error parsing contacts: {text[:200]}"
output_var: csv_output
output: "{csv_output}"
```
**Usage:**
```bash
cat business_cards.txt | extract-contacts
cat meeting_notes.txt | extract-contacts > contacts.csv
```
---
### sql-from-text
Generate SQL from natural language.
```yaml
name: sql-from-text
description: Generate SQL from natural language
arguments:
- flag: --dialect
variable: dialect
default: PostgreSQL
steps:
- type: prompt
prompt: |
Generate a {dialect} SQL query for this request. Output only the SQL, no explanation:
{input}
provider: claude-haiku
output_var: response
output: "{response}"
```
**Usage:**
```bash
echo "get all users who signed up last month" | sql-from-text
echo "count orders by status" | sql-from-text --dialect MySQL
```
---
### safe-sql
Generate SQL with safety checks.
```yaml
name: safe-sql
description: Generate SQL with safety checks
arguments:
- flag: --dialect
variable: dialect
default: PostgreSQL
steps:
- type: prompt
prompt: |
Generate a {dialect} SELECT query for: {input}
Output ONLY the SQL query, no explanation.
provider: claude-haiku
output_var: raw_sql
- type: code
code: |
import re
sql = raw_sql.strip()
sql = re.sub(r'^```sql?\s*', '', sql)
sql = re.sub(r'\s*```$', '', sql)
dangerous = ['DROP', 'DELETE', 'TRUNCATE', 'UPDATE', 'INSERT', 'ALTER', 'CREATE', 'GRANT']
warnings = []
for keyword in dangerous:
if re.search(r'\b' + keyword + r'\b', sql, re.I):
warnings.append(f"WARNING: Contains {keyword}")
if warnings:
validated = '\n'.join(warnings) + '\n\n' + sql
else:
validated = sql
output_var: validated
output: "{validated}"
```
**Usage:**
```bash
echo "get active users" | safe-sql
echo "remove inactive accounts" | safe-sql # Will show WARNING
```
---
### parse-log
Summarize and analyze log files.
```yaml
name: parse-log
description: Summarize and analyze log files
arguments:
- flag: --focus
variable: focus
default: errors and warnings
steps:
- type: prompt
prompt: |
Analyze these logs focusing on {focus}. Summarize key issues and patterns:
{input}
provider: claude-haiku
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat app.log | parse-log
tail -1000 /var/log/syslog | parse-log --focus "security events"
```
---
### csv-insights
Analyze CSV with sampled data for large files.
```yaml
name: csv-insights
description: Analyze CSV with sampled data for large files
arguments:
- flag: --question
variable: question
default: What patterns and insights can you find?
steps:
- type: code
code: |
import random
lines = input.strip().split('\n')
header = lines[0] if lines else ''
data_lines = lines[1:] if len(lines) > 1 else []
if len(data_lines) > 50:
sampled = random.sample(data_lines, 50)
sample_note = f"(Sampled 50 of {len(data_lines)} rows)"
else:
sampled = data_lines
sample_note = f"({len(data_lines)} rows)"
sampled_csv = header + '\n' + '\n'.join(sampled)
stats = f"Columns: {len(header.split(','))}, Rows: {len(data_lines)} {sample_note}"
output_var: sampled_csv,stats
- type: prompt
prompt: |
Analyze this CSV data. {stats}
Question: {question}
Data:
{sampled_csv}
provider: claude-haiku
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat sales.csv | csv-insights
cat large_data.csv | csv-insights --question "What are the top trends?"
```
---
## Advanced Multi-Step Tools
### log-errors
Extract and explain errors from large log files.
```yaml
name: log-errors
description: Extract and explain errors from large log files
arguments: []
steps:
- type: code
code: |
import re
lines = input.split('\n')
errors = [l for l in lines if re.search(r'\b(ERROR|CRITICAL|FATAL|Exception|Traceback)\b', l, re.I)]
result = []
for i, line in enumerate(lines):
if re.search(r'\b(ERROR|CRITICAL|FATAL|Exception|Traceback)\b', line, re.I):
result.extend(lines[i:i+5])
extracted = '\n'.join(result[:200])
output_var: extracted
- type: prompt
prompt: |
Analyze these error log entries. Group by error type, explain likely causes, and suggest fixes:
{extracted}
provider: claude-haiku
output_var: response
output: "{response}"
```
**Usage:**
```bash
cat huge_app.log | log-errors
zcat archived.log.gz | log-errors
```
---
### diff-focus
Review only the added/changed code in a diff.
```yaml
name: diff-focus
description: Review only the added/changed code in a diff
arguments: []
steps:
- type: code
code: |
lines = input.split('\n')
result = []
for i, line in enumerate(lines):
if line.startswith('@@') or line.startswith('+++') or line.startswith('---'):
result.append(line)
elif line.startswith('+') and not line.startswith('+++'):
result.append(line)
extracted = '\n'.join(result)
output_var: extracted
- type: prompt
prompt: |
Review these added lines of code. Focus on bugs, security issues, and improvements:
{extracted}
provider: claude-haiku
output_var: response
output: "{response}"
```
**Usage:**
```bash
git diff | diff-focus
git diff HEAD~5 | diff-focus
```
---
### changelog
Generate changelog from git commits.
```yaml
name: changelog
description: Generate changelog from git commits
arguments:
- flag: --since
variable: since
default: last week
steps:
- type: code
code: |
lines = input.strip().split('\n')
commits = []
for line in lines:
if line.strip() and not line.startswith('commit '):
commits.append(line.strip())
git_log = '\n'.join(commits[:100])
output_var: git_log
- type: prompt
prompt: |
Generate a user-friendly changelog from these git commits. Group by category (Features, Fixes, Improvements, etc). Use markdown with bullet points:
{git_log}
provider: opencode-pickle
output_var: changelog_raw
- type: code
code: |
from datetime import datetime
header = f"""# Changelog
Generated: {datetime.now().strftime('%Y-%m-%d')}
"""
formatted = header + changelog_raw
output_var: formatted
output: "{formatted}"
```
**Usage:**
```bash
git log --oneline -50 | changelog
git log --since="2024-01-01" --oneline | changelog
```
---
### code-validate
Generate and validate Python code.
```yaml
name: code-validate
description: Generate and validate Python code
arguments:
- flag: --task
variable: task
default: ""
steps:
- type: prompt
prompt: |
Write Python code to: {task}
Context/input data:
{input}
Output ONLY the Python code, no markdown, no explanation.
provider: claude-haiku
output_var: raw_code
- type: code
code: |
import ast
import re
code = raw_code.strip()
code = re.sub(r'^```python?\s*', '', code)
code = re.sub(r'\s*```$', '', code)
try:
ast.parse(code)
validated = code
except SyntaxError as e:
validated = f"# SYNTAX ERROR at line {e.lineno}: {e.msg}\n# Fix needed:\n\n{code}"
output_var: validated
output: "{validated}"
```
**Usage:**
```bash
echo "list of numbers: 1,2,3,4,5" | code-validate --task "calculate the sum"
cat data.json | code-validate --task "parse and find the maximum value"
```
---
## Writing Your Own Tools
### Basic Structure
```yaml
name: my-tool
description: What this tool does
arguments:
- flag: --option
variable: option_name
default: "default value"
steps:
- type: prompt
prompt: |
Your prompt here with {input} and {option_name}
provider: opencode-pickle
output_var: response
output: "{response}"
```
### Multi-Step Structure
```yaml
name: my-pipeline
description: Multi-step tool
steps:
# Step 1: Preprocess with code
- type: code
code: |
# Python code here
processed = input.upper()
output_var: processed
# Step 2: AI processing
- type: prompt
prompt: "Analyze: {processed}"
provider: claude-haiku
output_var: analysis
# Step 3: Post-process with code
- type: code
code: |
result = f"RESULT:\n{analysis}"
output_var: result
output: "{result}"
```
### Tips
1. **Use cheap providers** for simple tasks (`opencode-pickle`)
2. **Use code steps** to validate AI output
3. **Preprocess large inputs** to reduce tokens
4. **Test with mock** before using real AI: `--provider mock`
+247
View File
@@ -0,0 +1,247 @@
# Installation Guide
Complete installation instructions for SmartTools.
## Quick Install
```bash
pip install smarttools
export PATH="$HOME/.local/bin:$PATH"
smarttools refresh
```
## Detailed Installation
### Step 1: Install Python 3.10+
**Ubuntu/Debian:**
```bash
sudo apt update
sudo apt install python3 python3-pip
```
**macOS:**
```bash
brew install [email protected]
```
**Verify:**
```bash
python3 --version # Should be 3.10 or higher
```
### Step 2: Install SmartTools
**From PyPI (recommended):**
```bash
pip install smarttools
```
**From source (for development):**
```bash
git clone https://github.com/yourusername/smarttools.git
cd smarttools
pip install -e .
```
**With TUI support (recommended):**
```bash
pip install smarttools urwid
```
### Step 3: Configure PATH
SmartTools creates executable wrappers in `~/.local/bin/`. Add this to your shell config:
**Bash (~/.bashrc):**
```bash
export PATH="$HOME/.local/bin:$PATH"
```
**Zsh (~/.zshrc):**
```bash
export PATH="$HOME/.local/bin:$PATH"
```
**Fish (~/.config/fish/config.fish):**
```fish
set -gx PATH $HOME/.local/bin $PATH
```
Then reload:
```bash
source ~/.bashrc # or restart terminal
```
### Step 4: Install an AI Provider
You need at least one AI CLI tool. Choose based on your needs:
#### Free Options
**OpenCode (recommended for free):**
```bash
# Install
curl -fsSL https://opencode.ai/install | bash
# Authenticate (uses browser)
~/.opencode/bin/opencode auth
```
#### Paid Options
**Claude CLI:**
```bash
# Install
npm install -g @anthropic-ai/claude-cli
# Authenticate
claude auth
```
**Gemini CLI:**
```bash
# Install
pip install google-generativeai
# Or the CLI wrapper
npm install -g @anthropic-ai/gemini-cli
```
**Codex (OpenAI):**
```bash
npm install -g codex-cli
codex auth
```
See [PROVIDERS.md](PROVIDERS.md) for full provider setup instructions.
### Step 5: Verify Installation
```bash
# Check SmartTools is installed
smarttools --version
# List available providers
smarttools providers
# Test with mock provider (no AI needed)
echo "hello world" | smarttools run --provider mock
```
### Step 6: Create Wrapper Scripts
After installation or any Python environment change:
```bash
smarttools refresh
```
This creates executable scripts in `~/.local/bin/` for all your tools.
## Installing Example Tools
### Quick Install (All 28 Tools)
```bash
curl -sSL https://raw.githubusercontent.com/yourusername/smarttools/main/examples/install.py | python3
smarttools refresh
```
### Manual Install
Copy individual tool configs to `~/.smarttools/<toolname>/config.yaml`.
See [EXAMPLES.md](EXAMPLES.md) for all tool configurations.
## Troubleshooting
### "smarttools: command not found"
PATH is not configured. Add to your shell config:
```bash
export PATH="$HOME/.local/bin:$PATH"
```
### "Provider 'X' not found"
1. Check provider is installed: `which claude` or `which opencode`
2. Check provider is configured: `smarttools providers`
3. Add missing provider: `smarttools providers add`
### "Permission denied" on tool execution
Refresh wrapper scripts:
```bash
smarttools refresh
```
### TUI doesn't start / looks broken
Install urwid:
```bash
pip install urwid
```
Ensure your terminal supports mouse (most modern terminals do).
### Tools work with `smarttools run` but not directly
The wrapper scripts may be outdated. Refresh them:
```bash
smarttools refresh
```
### Python version mismatch
If you have multiple Python versions, ensure SmartTools uses the right one:
```bash
python3.11 -m pip install smarttools
```
## Uninstalling
```bash
# Remove package
pip uninstall smarttools
# Remove config and tools (optional)
rm -rf ~/.smarttools
# Remove wrapper scripts (optional)
rm ~/.local/bin/{summarize,translate,fix-grammar,...}
```
## Upgrading
```bash
pip install --upgrade smarttools
smarttools refresh
```
## Directory Structure
After installation:
```
~/.smarttools/
├── providers.yaml # Provider configurations
├── summarize/
│ └── config.yaml # Tool config
├── translate/
│ ├── config.yaml
│ └── prompt.txt # External prompt file (optional)
└── ...
~/.local/bin/
├── smarttools # Main command
├── summarize # Tool wrapper script
├── translate # Tool wrapper script
└── ...
```
## Next Steps
1. [Set up providers](PROVIDERS.md)
2. [Browse example tools](EXAMPLES.md)
3. Run `smarttools ui` to create your first tool
+312
View File
@@ -0,0 +1,312 @@
# Provider Setup Guide
SmartTools works with any AI CLI tool that accepts input via stdin or arguments. This guide covers setup for the most popular providers.
## Provider Comparison
We profiled 12 providers with a 4-task benchmark (Math, Code, Reasoning, Data Extraction):
| Provider | Speed | Score | Cost | Best For |
|----------|-------|-------|------|----------|
| **opencode-deepseek** | 13s | 4/4 | ~$0.28/M tokens | **Best value** - daily driver |
| **opencode-pickle** | 13s | 4/4 | FREE | **Best free** - accurate |
| **claude-haiku** | 14s | 4/4 | ~$0.25/M tokens | Fast + high quality |
| **codex** | 14s | 4/4 | ~$1.25/M tokens | Reliable, auto-routes |
| **claude** | 18s | 4/4 | Varies | Auto-routes to best |
| **claude-opus** | 18s | 4/4 | ~$15/M tokens | Highest quality |
| **claude-sonnet** | 21s | 4/4 | ~$3/M tokens | Balanced |
| **opencode-nano** | 24s | 4/4 | Paid | GPT-5 Nano |
| **gemini-flash** | 28s | 4/4 | ~$0.075/M tokens | Google, faster |
| **opencode-reasoner** | 33s | 4/4 | ~$0.28/M tokens | Complex reasoning |
| **gemini** | 91s | 3/4 | ~$1.25/M tokens | 1M token context |
| **opencode-grok** | 11s | 2/4 | FREE | Fastest but unreliable |
### Recommendations
- **Daily use:** `opencode-deepseek` or `opencode-pickle` (free)
- **Quality work:** `claude-haiku` or `claude-opus`
- **Complex reasoning:** `opencode-reasoner`
- **Large documents:** `gemini` (1M token context window)
- **Budget:** `opencode-pickle` (free) or `opencode-deepseek` (cheap)
## Provider Setup
### OpenCode (Recommended)
OpenCode provides access to multiple models including free options.
**Install:**
```bash
curl -fsSL https://opencode.ai/install | bash
```
**Authenticate:**
```bash
~/.opencode/bin/opencode auth
```
**Available Models:**
| Provider Name | Model | Cost |
|---------------|-------|------|
| `opencode-deepseek` | deepseek-chat | Cheap |
| `opencode-pickle` | big-pickle | FREE |
| `opencode-grok` | grok-code | FREE |
| `opencode-nano` | gpt-5-nano | Paid |
| `opencode-reasoner` | deepseek-reasoner | Cheap |
**Test:**
```bash
echo "Hello" | ~/.opencode/bin/opencode run --model opencode/big-pickle
```
### Claude CLI
Anthropic's official CLI for Claude models.
**Install:**
```bash
npm install -g @anthropic-ai/claude-cli
# or
brew install claude
```
**Authenticate:**
```bash
claude auth
```
**Available Models:**
| Provider Name | Model | Cost |
|---------------|-------|------|
| `claude` | Auto-routes | Varies |
| `claude-haiku` | Haiku 4.5 | Cheap |
| `claude-sonnet` | Sonnet 4.5 | Medium |
| `claude-opus` | Opus 4.5 | Expensive |
**Test:**
```bash
echo "Hello" | claude -p
```
### Codex (OpenAI)
OpenAI's Codex CLI with auto-routing.
**Install:**
```bash
npm install -g codex-cli
```
**Authenticate:**
```bash
codex auth # Uses ChatGPT account
```
**Test:**
```bash
echo "Hello" | codex exec -
```
### Gemini
Google's Gemini models. Best for large context (1M tokens).
**Install:**
```bash
npm install -g @anthropic-ai/gemini-cli
# or
pip install google-generativeai
```
**Authenticate:**
```bash
gemini auth # Uses Google account
```
**Available Models:**
| Provider Name | Model | Notes |
|---------------|-------|-------|
| `gemini` | gemini-2.5-pro | Quality, slow CLI |
| `gemini-flash` | gemini-2.5-flash | Faster |
**Note:** Gemini CLI has known performance issues. Use `gemini-flash` for interactive tasks, `gemini` for large documents.
**Test:**
```bash
echo "Hello" | gemini --model gemini-2.5-flash
```
## Managing Providers
### List Providers
```bash
smarttools providers
```
### Add Custom Provider
```bash
smarttools providers add
```
Or edit `~/.smarttools/providers.yaml`:
```yaml
providers:
- name: my-custom
command: my-ai-tool --prompt
description: My custom AI tool
```
### Provider Command Format
The command should:
1. Accept input via stdin
2. Output response to stdout
3. Exit 0 on success
Example commands:
```bash
# Claude
claude -p
# OpenCode
$HOME/.opencode/bin/opencode run --model deepseek/deepseek-chat
# Gemini
gemini --model gemini-2.5-flash
# Codex
codex exec -
# Custom (any tool that reads stdin)
my-tool --input -
```
### Environment Variables
Provider commands can use environment variables:
```yaml
providers:
- name: opencode
command: $HOME/.opencode/bin/opencode run
```
`$HOME` and `~` are expanded automatically.
## Using Providers in Tools
### In Tool Config
```yaml
steps:
- type: prompt
prompt: "Summarize: {input}"
provider: opencode-pickle # Use this provider
output_var: response
```
### Override at Runtime
```bash
# Use a different provider for this run
cat file.txt | summarize --provider claude-opus
```
### Provider Selection Strategy
1. **Tool default** - Set in tool's config.yaml
2. **Runtime override** - `--provider` flag
3. **Cost optimization** - Use cheap providers for simple tasks
4. **Quality needs** - Use opus/sonnet for important work
## Troubleshooting
### "Provider 'X' not found"
1. Check it's in your providers list: `smarttools providers`
2. Verify the command works: `echo "test" | <command>`
3. Add it: `smarttools providers add`
### "Command 'X' not found"
The AI CLI tool isn't installed or not in PATH:
```bash
which claude # Should show path
which opencode # Might need full path
```
For OpenCode, use full path in provider:
```yaml
command: $HOME/.opencode/bin/opencode run
```
### Slow Provider
- Use `gemini-flash` instead of `gemini`
- Use `claude-haiku` instead of `claude-opus`
- Use `opencode-deepseek` for best speed/quality ratio
### Provider Errors
Check the provider works directly:
```bash
echo "Say hello" | claude -p
echo "Say hello" | ~/.opencode/bin/opencode run
```
If it works directly but not in SmartTools, check:
1. Provider command in `~/.smarttools/providers.yaml`
2. Environment variables are expanded correctly
## Cost Optimization
### Free Providers
- `opencode-pickle` - Big Pickle model (FREE, accurate)
- `opencode-grok` - Grok Code (FREE, fast but less reliable)
### Cheap Providers
- `opencode-deepseek` - ~$0.28/M tokens
- `opencode-reasoner` - ~$0.28/M tokens
- `claude-haiku` - ~$0.25/M tokens
### Tips
1. Use `opencode-pickle` for simple tasks (free + accurate)
2. Use `claude-haiku` when you need reliability
3. Reserve `claude-opus` for important work
4. Use `gemini` only for large document analysis
## Adding New Providers
Any CLI tool that:
- Reads from stdin
- Writes to stdout
- Exits 0 on success
Can be a provider. Examples:
**Local LLM (Ollama):**
```yaml
- name: ollama-llama
command: ollama run llama3
description: Local Llama 3
```
**Custom API wrapper:**
```yaml
- name: my-api
command: curl -s -X POST https://my-api.com/chat -d @-
description: My custom API
```
**Python script:**
```yaml
- name: my-python
command: python3 ~/scripts/my_ai.py
description: Custom Python AI
```