rob/CmdForge
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add meta-tools, collections, and attribution features

Browse Source 
Meta-tools:
- Add ToolStep dataclass for tools calling other tools
- Implement execute_tool_step() with recursion depth limiting
- Add check_dependencies() for validating tool dependencies
- Add 'cmdforge check' command for dependency verification

Collections:
- Add collections API endpoints and database schema
- Create collections web UI (list and detail pages)
- Add collections to navigation header and homepage
- Document collections in REGISTRY.md

Attribution:
- Add source attribution fields to tool detail page
- Document source types (original, adapted, imported)
- Add license field documentation

Documentation updates across README, DESIGN.md, REGISTRY.md,
and new META_TOOLS.md design document.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
This commit is contained in:
rob
2026-01-03 03:05:23 -04:00
co-authored by Claude Opus 4.5
parent 497fe87fc5
commit 2c139b3982
20 changed files with 1299 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/COLLECTIONS.md
+116
View File
@@ -0,0 +1,116 @@
# CmdForge Collections
Collections are curated groups of tools that can be installed together with a single command.
## Use Cases
1. **Thematic bundles**: "writing-toolkit" with grammar, tone, simplify tools
2. **Application stacks**: "data-science" with json-extract, csv-insights, etc.
3. **Source bundles**: "fabric-text" with all Fabric text processing patterns
4. **Workflow packages**: Tools that work well together for a specific task
## Collection Manifest Format
Collections are defined in the registry repo under `collections/`:
```yaml
# collections/writing-toolkit.yaml
name: writing-toolkit
display_name: Writing Toolkit
description: Essential tools for writers and editors
icon: pencil # Optional icon identifier
maintainer: official # Publisher who maintains this collection
tools:
- official/fix-grammar
- official/simplify
- official/tone-shift
- official/expand
- official/proofread
# Optional: version constraints
pinned:
official/fix-grammar: "^1.0.0"
# Optional: suggested order for documentation
order:
- official/fix-grammar
- official/proofread
- official/simplify
- official/tone-shift
- official/expand
tags:
- writing
- editing
- grammar
```
## CLI Usage
```bash
# List available collections
cmdforge collections list
# View collection details
cmdforge collections info writing-toolkit
# Install all tools in a collection
cmdforge collections install writing-toolkit
# Install with version constraints from collection
cmdforge collections install writing-toolkit --pinned
```
## API Endpoints
```
GET /api/v1/collections # List all collections
GET /api/v1/collections/:name # Get collection details with tool info
GET /api/v1/collections/:name/tools # Get just the tools list
```
## Database Schema
```sql
CREATE TABLE collections (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT UNIQUE NOT NULL,
display_name TEXT NOT NULL,
description TEXT,
icon TEXT,
maintainer TEXT NOT NULL,
tools TEXT NOT NULL, -- JSON array of tool refs
pinned TEXT, -- JSON object of version constraints
tags TEXT, -- JSON array
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_collections_name ON collections(name);
CREATE INDEX idx_collections_maintainer ON collections(maintainer);
```
## Web UI
- `/collections` - Browse all collections
- `/collections/:name` - Collection detail page with tool grid
- Install button that shows CLI command
- Filter by tag, maintainer
## Sync from Registry Repo
Collections sync from the registry repo just like tools:
```
CmdForge-Registry/
├── tools/
│ └── ...
└── collections/
├── writing-toolkit.yaml
├── data-science.yaml
├── fabric-text.yaml
└── fabric-code.yaml
```
The webhook sync process reads `collections/*.yaml` and upserts to database.
docs/DESIGN.md
+17
View File
@@ -46,6 +46,11 @@ steps:
provider: claude
output_var: response
output: "{response}"
# Optional: declare dependencies for meta-tools
dependencies:
- official/summarize
- official/translate@^1.0.0 # With version constraint
```
### Step Types
@@ -68,6 +73,17 @@ output: "{response}"
code_file: processed.py # Optional: external file storage
```
**Tool Step** - Calls another CmdForge tool (meta-tools):
```yaml
- type: tool
tool: official/summarize # Tool reference (owner/name or just name for local)
input: "{input}" # Input to pass (supports variable substitution)
args: # Optional arguments
max_words: "100"
output_var: summary # Variable to store the tool's output
provider: claude # Optional: override the called tool's provider
```
Steps execute in order. Each step's `output_var` becomes available to subsequent steps.
### Variables
@@ -168,6 +184,7 @@ cmdforge delete sum # Delete tool
cmdforge test sum # Test with mock provider
cmdforge run sum # Run tool for real
cmdforge refresh # Refresh all wrapper scripts
cmdforge check sum # Check dependencies for meta-tools
cmdforge ui # Launch interactive UI
```
docs/META_TOOLS.md
+249
View File
@@ -0,0 +1,249 @@
# Meta-Tools: Tools That Call Other Tools
Meta-tools are CmdForge tools that can invoke other tools as steps in their workflow. This enables powerful composition and reuse of existing tools.
## Step Type: `tool`
A new step type `tool` allows calling another CmdForge tool from within a tool's workflow.
### Manifest Format
```yaml
name: summarize-and-translate
description: Summarize text then translate the summary
version: 1.0.0
category: Text
steps:
- type: tool
tool: official/summarize # Tool to call (owner/name or just name for local)
input: "{input}" # What to pass as input (supports variable substitution)
args: # Optional arguments to pass to the tool
max_words: "100"
output_var: summary # Variable to store the output
- type: tool
tool: official/translate
input: "{summary}" # Use output from previous step
args:
target_language: "{language}" # Can use tool's own arguments
output_var: translated
output: "{translated}"
arguments:
- flag: "--language"
variable: language
default: "Spanish"
description: "Target language for translation"
```
### Resolution Order
When resolving a tool reference in a `tool` step:
1. **Fully qualified**: `owner/name` - looks up in registry or installed tools
2. **Local tool**: Just `name` - checks local `~/.cmdforge/<name>/config.yaml` first
3. **Registry fallback**: If not found locally, checks installed registry tools
### Tool Step Properties
| Property | Required | Description |
|----------|----------|-------------|
| `type` | Yes | Must be `"tool"` |
| `tool` | Yes | Tool reference (owner/name or name) |
| `input` | No | Input to pass to the tool (default: current `{input}`) |
| `args` | No | Dictionary of arguments to pass |
| `output_var` | Yes | Variable name to store the tool's output |
| `provider` | No | Override provider for the called tool |
### Execution
When a `tool` step executes:
1. Resolve the tool reference to find the tool definition
2. Substitute variables in `input` and `args` values
3. Execute the tool with the resolved input and arguments
4. Capture the output in the specified `output_var`
### Error Handling
- If a called tool fails, the parent tool fails with an appropriate error
- Tool not found errors include helpful installation suggestions
- Circular dependencies are detected and prevented
## Dependency Resolution
### Declaring Dependencies
Tools can declare their dependencies in the config:
```yaml
name: my-meta-tool
dependencies:
- official/summarize
- official/translate@^1.0.0 # With version constraint
```
### CLI Integration
```bash
# Install a tool and its dependencies
cmdforge install official/summarize-and-translate
# Check if dependencies are satisfied
cmdforge check my-meta-tool
# Install missing dependencies
cmdforge install --deps my-meta-tool
```
### Dependency Checking
Before running a meta-tool, the runner verifies all dependencies are installed:
```python
def check_dependencies(tool: Tool) -> List[str]:
"""Returns list of missing dependencies."""
missing = []
for dep in tool.dependencies:
if not is_tool_installed(dep):
missing.append(dep)
return missing
```
## Security Considerations
1. **Sandboxing**: Called tools run in the same process context as the parent
2. **Input validation**: All inputs are treated as untrusted strings
3. **No shell execution**: Tool references cannot contain shell commands
4. **Depth limit**: Maximum nesting depth (default: 10) prevents runaway recursion
## Example: Multi-Step Analysis Pipeline
```yaml
name: code-review-pipeline
description: Comprehensive code review using multiple specialized tools
version: 1.0.0
category: Developer
dependencies:
- official/analyze-complexity
- official/find-bugs
- official/suggest-improvements
steps:
# Step 1: Analyze code complexity
- type: tool
tool: official/analyze-complexity
input: "{input}"
output_var: complexity_report
# Step 2: Find potential bugs
- type: tool
tool: official/find-bugs
input: "{input}"
output_var: bug_report
# Step 3: Generate improvement suggestions based on findings
- type: tool
tool: official/suggest-improvements
input: |
## Code:
{input}
## Complexity Analysis:
{complexity_report}
## Bug Report:
{bug_report}
output_var: suggestions
output: |
# Code Review Results
## Complexity Analysis
{complexity_report}
## Potential Issues
{bug_report}
## Improvement Suggestions
{suggestions}
```
## Implementation Notes
### ToolStep Dataclass
```python
@dataclass
class ToolStep:
"""A step that calls another tool."""
tool: str # Tool reference (owner/name or name)
output_var: str # Variable to store output
input_template: str = "{input}" # Input template
args: Dict[str, str] = field(default_factory=dict)
provider: Optional[str] = None # Provider override
def to_dict(self) -> dict:
d = {
"type": "tool",
"tool": self.tool,
"output_var": self.output_var,
}
if self.input_template != "{input}":
d["input"] = self.input_template
if self.args:
d["args"] = self.args
if self.provider:
d["provider"] = self.provider
return d
```
### Execution Function
```python
def execute_tool_step(
step: ToolStep,
variables: dict,
depth: int = 0,
max_depth: int = 10
) -> tuple[str, bool]:
"""Execute a tool step by calling another tool."""
if depth >= max_depth:
return "", False # Prevent infinite recursion
# Resolve the tool
try:
resolved = resolve_tool(step.tool)
except ToolNotFoundError:
print(f"Error: Tool '{step.tool}' not found", file=sys.stderr)
return "", False
# Prepare input
input_text = substitute_variables(step.input_template, variables)
# Prepare arguments
args = {}
for key, value in step.args.items():
args[key] = substitute_variables(value, variables)
# Run the tool
output, exit_code = run_tool(
tool=resolved.tool,
input_text=input_text,
custom_args=args,
provider_override=step.provider,
_depth=depth + 1 # Track nesting depth
)
return output, exit_code == 0
```
## Future Enhancements
- **Parallel tool execution**: Run independent tool steps concurrently
- **Conditional execution**: Skip steps based on conditions
- **Tool aliases**: Define shorthand names for frequently used tools
- **Caching**: Cache tool outputs for identical inputs
docs/REGISTRY.md
+127
View File
@@ -36,6 +36,8 @@ Core API endpoints:
- `GET /api/v1/tools/{owner}/{name}/download?version=...`
- `POST /api/v1/tools` (publish)
- `GET /api/v1/categories`
- `GET /api/v1/collections`
- `GET /api/v1/collections/{name}`
- `GET /api/v1/stats/popular`
- `POST /api/v1/webhook/gitea`
@@ -239,6 +241,124 @@ registry:
downloads: 142
```
### Attribution and Source Fields
Tools can include optional source attribution for provenance and licensing:
```yaml
name: summarize
version: "1.2.0"
description: "Summarize text using AI"
# Attribution fields (optional)
source:
type: original # original, adapted, or imported
license: MIT # SPDX license identifier
url: https://example.com/tool-repo
author: "Original Author"
# For adapted/imported tools
original_tool: other/[email protected]
changes: "Added French language support"
```
**Source types:**
| Type | Description |
|------|-------------|
| `original` | Created from scratch by the publisher |
| `adapted` | Based on another tool with modifications |
| `imported` | Direct import of external tool (e.g., from npm/pip) |
**License field:**
- Uses SPDX identifiers: `MIT`, `Apache-2.0`, `GPL-3.0`, etc.
- Required for registry publication
- Validated against SPDX license list
## Collections
Collections are curated groups of tools that can be installed together with a single command.
### Collection Structure
Collections are defined in `collections/{name}.yaml`:
```yaml
name: text-processing-essentials
display_name: "Text Processing Essentials"
description: "Essential tools for text processing and manipulation"
icon: "📝"
tools:
- official/summarize
- official/translate
- official/fix-grammar
- official/simplify
- official/tone-shift
# Optional
curator: official
tags: ["text", "nlp", "writing"]
```
### Collections API
**List all collections:**
```
GET /api/v1/collections
Response:
{
"data": [
{
"name": "text-processing-essentials",
"display_name": "Text Processing Essentials",
"description": "Essential tools for text processing...",
"icon": "📝",
"tool_count": 5,
"curator": "official"
}
],
"meta": {"page": 1, "per_page": 20, "total": 8}
}
```
**Get collection details:**
```
GET /api/v1/collections/{name}
Response:
{
"data": {
"name": "text-processing-essentials",
"display_name": "Text Processing Essentials",
"description": "Essential tools for text processing...",
"icon": "📝",
"curator": "official",
"tools": [
{"owner": "official", "name": "summarize", "version": "1.2.0", ...},
{"owner": "official", "name": "translate", "version": "2.1.0", ...}
]
}
}
```
### CLI Commands
```bash
# List available collections
cmdforge registry collections
# View collection details
cmdforge registry collections text-processing-essentials
# Install all tools in a collection
cmdforge registry install --collection text-processing-essentials
# Show what would be installed (dry run)
cmdforge registry install --collection text-processing-essentials --dry-run
```
**Schema compatibility note:** The current CmdForge config parser may reject unknown top-level keys like `deprecated`, `replacement`, and `registry`. Before implementing registry features:
1. Update the YAML parser to ignore unknown keys (permissive mode)
2. Or explicitly define these fields in the Tool dataclass with defaults
@@ -1151,6 +1271,11 @@ CmdForge-Registry/
├── categories/
│ └── categories.yaml # Category definitions
│
├── collections/ # Curated tool collections
│ ├── text-processing-essentials.yaml
│ ├── developer-toolkit.yaml
│ └── data-pipeline-basics.yaml
│
├── index.json # Auto-generated search index
│
├── .gitea/
@@ -1736,6 +1861,8 @@ cmdforge.brrd.tech (or gitea.brrd.tech/registry)
├── /tools/{owner}/{name} # Tool detail page
├── /categories # Browse by category
├── /categories/{name} # Tools in category
├── /collections # Browse curated collections
├── /collections/{name} # Collection detail page
├── /search?q=... # Search results
├── /docs # Documentation
│ ├── /docs/getting-started