Add strip_fences option and fix exec() scoping for nested functions
- Add strip_fences field to PromptStep for removing markdown code fences - Fix exec() to use same dict for globals and locals, allowing nested functions to access module-level imports - Add GUI checkbox for strip_fences option - Update CLAUDE.md with improved documentation Co-Authored-By: Claude Opus 4.5 <[email protected]>
This commit is contained in:
@@ -12,11 +12,24 @@ CmdForge is a lightweight personal tool builder for AI-powered CLI commands. It
|
||||
# Install for development
|
||||
pip install -e ".[dev]"
|
||||
|
||||
# Run tests
|
||||
pytest
|
||||
# Run all unit tests (excluding integration tests that need a server)
|
||||
pytest tests/ -m "not integration"
|
||||
|
||||
# Run a single test
|
||||
pytest tests/test.py::test_name
|
||||
# Run all tests with verbose output
|
||||
pytest tests/ -v
|
||||
|
||||
# Run a specific test file
|
||||
pytest tests/test_runner.py -v
|
||||
|
||||
# Run a specific test class
|
||||
pytest tests/test_runner.py::TestSubstituteVariables -v
|
||||
|
||||
# Run with coverage
|
||||
pytest tests/ --cov=cmdforge --cov-report=html
|
||||
|
||||
# Run integration tests (requires local registry server at localhost:5000)
|
||||
python -m cmdforge.registry.app # Start server first
|
||||
pytest tests/test_registry_integration.py -v -m integration
|
||||
|
||||
# Run the CLI
|
||||
python -m cmdforge.cli
|
||||
@@ -29,14 +42,19 @@ cmdforge
|
||||
|
||||
### Core Modules (`src/cmdforge/`)
|
||||
|
||||
- **cli/**: CLI commands entry points (`cmdforge` command). Routes subcommands: list, create, edit, delete, test, run, refresh, collections
|
||||
- **tool.py**: Tool definition dataclasses (`Tool`, `ToolArgument`, `PromptStep`, `CodeStep`), YAML config loading/saving, wrapper script generation
|
||||
- **cli/**: CLI commands entry points (`cmdforge` command). Routes subcommands: list, create, edit, delete, test, run, refresh, collections, registry, settings, system-deps
|
||||
- **tool.py**: Tool definition dataclasses (`Tool`, `ToolArgument`, `PromptStep`, `CodeStep`, `ToolStep`), YAML config loading/saving, wrapper script generation
|
||||
- **runner.py**: Execution engine. Runs tool steps sequentially, handles variable substitution (`{input}`, `{varname}`), executes Python code steps via `exec()`, handles nested tool calls with depth limit (MAX_TOOL_DEPTH=10)
|
||||
- **resolver.py**: Tool resolution. `resolve_tool()` searches: project manifest → local tools → owner/name → registry. Returns `ResolvedTool` with path info
|
||||
- **collection.py**: Collection management (`Collection` dataclass, `resolve_tool_references()`, `classify_tool_reference()`), local collection storage in `~/.cmdforge/collections/`
|
||||
- **runner.py**: Execution engine. Runs tool steps sequentially, handles variable substitution (`{input}`, `{varname}`), executes Python code steps via `exec()`
|
||||
- **providers.py**: Provider abstraction. Calls AI CLI tools via subprocess, reads provider configs from `~/.cmdforge/providers.yaml`
|
||||
- **profiles.py**: AI persona profiles with system prompts, stored in `~/.cmdforge/profiles/`
|
||||
- **manifest.py**: Project manifest (`cmdforge.yaml`) for declaring tool dependencies with version constraints
|
||||
- **lockfile.py**: Lock file support for reproducible installs (`cmdforge.lock`)
|
||||
- **registry_client.py**: Client for registry API (search, publish, download, authentication)
|
||||
- **gui/**: PySide6 desktop GUI
|
||||
- **main_window.py**: Main application window with sidebar navigation
|
||||
- **pages/**: Tools page, Tool Builder, Registry browser, Collections page, Providers management
|
||||
- **pages/**: Tools page, Tool Builder, Registry browser, Collections page, Providers management, Profiles
|
||||
- **dialogs/**: Step editors, Argument editor, Provider dialog, Connect/Publish dialogs
|
||||
|
||||
### Key Paths
|
||||
@@ -58,8 +76,9 @@ Tools are YAML configs with:
|
||||
|
||||
### Step Types
|
||||
|
||||
1. **Prompt Step**: Calls AI provider with template, stores result in `output_var`
|
||||
2. **Code Step**: Executes Python code via `exec()`, captures specified variables
|
||||
1. **Prompt Step**: Calls AI provider with template, stores result in `output_var`. Supports `profile` for AI personas and `strip_fences` for markdown cleanup
|
||||
2. **Code Step**: Executes Python code via `exec()`, captures specified variables (comma-separated for multiple outputs)
|
||||
3. **Tool Step**: Calls another tool (meta-tools), supports `args` dict and `provider` override. Dependencies resolved via `resolver.py`
|
||||
|
||||
### Variable Flow
|
||||
|
||||
@@ -102,8 +121,10 @@ CmdForge includes a web interface and tool registry:
|
||||
|
||||
### Registry Modules (`src/cmdforge/registry/`)
|
||||
|
||||
- **app.py**: Registry API (tool publishing, search, downloads)
|
||||
- **db.py**: SQLite schema and queries
|
||||
- **app.py**: Flask-based Registry API (tool publishing, search, downloads, authentication, rate limiting)
|
||||
- **db.py**: SQLite schema and queries (`connect_db()`, `query_one()`, `query_all()`)
|
||||
- **embeddings.py**: Semantic search with vector embeddings for tool discovery
|
||||
- **sync.py**: Git-based tool sync from Gitea repository
|
||||
|
||||
### Key URLs
|
||||
|
||||
@@ -122,6 +143,19 @@ python -m cmdforge.web.app
|
||||
CMDFORGE_REGISTRY_DB=/path/to/db PORT=5050 python -m cmdforge.web.app
|
||||
```
|
||||
|
||||
## Testing Conventions
|
||||
|
||||
Tests use `pytest` with common fixtures:
|
||||
- `temp_tools_dir(tmp_path)`: Redirects `TOOLS_DIR` and `BIN_DIR` to temp directory
|
||||
- `temp_providers_file(tmp_path)`: Redirects `providers.yaml` to temp directory
|
||||
|
||||
Mocking strategy:
|
||||
- **File system**: Use `tmp_path` fixture and patch module-level paths
|
||||
- **Subprocess calls**: Mock `subprocess.run` and `shutil.which`
|
||||
- **Provider calls**: Mock `call_provider` to return `ProviderResult`
|
||||
|
||||
Integration tests are marked with `@pytest.mark.integration` and require a running registry server.
|
||||
|
||||
## Infrastructure Documentation
|
||||
|
||||
For deployment, server details, and operations, see the `docs/` folder:
|
||||
|
||||
Reference in New Issue
Block a user