Add provider fallback, visual schema builder, and UI improvements

Provider fallback system:
- Add optional fallback field to Provider dataclass
- Automatically try fallback provider when primary fails (timeout, error, offline)
- Prevent infinite loops with circular fallback detection
- Add fallback column and dropdown in providers UI

Visual schema builder for prompt steps:
- Add SchemaBuilderDialog with table-based field editor
- Support field name, type, description, and required flag
- Live JSON schema preview
- Integrate into PromptStepDialog with Edit Schema button
- Fix output_schema being lost when editing prompt steps

Other improvements:
- Add search box to tools page
- Improve schema example generation for structured output
- Update CLAUDE.md documentation

Co-Authored-By: Claude Opus 4.5 <[email protected]>
This commit is contained in:
rob
2026-02-21 00:50:18 -04:00
co-authored by Claude Opus 4.5
parent 8581775002
commit 71c6358b9d
9 changed files with 892 additions and 99 deletions
+25 -17
View File
@@ -15,14 +15,12 @@ pip install -e ".[dev]"
# Run all unit tests (excluding integration tests that need a server)
pytest tests/ -m "not integration"
# Run all tests with verbose output
pytest tests/ -v
# Run a specific test file
pytest tests/test_runner.py -v
# Run a specific test class
# Run a specific test class or method
pytest tests/test_runner.py::TestSubstituteVariables -v
pytest tests/test_runner.py::TestSubstituteVariables::test_simple_substitution -v
# Run with coverage
pytest tests/ --cov=cmdforge --cov-report=html
@@ -31,11 +29,10 @@ pytest tests/ --cov=cmdforge --cov-report=html
python -m cmdforge.registry.app # Start server first
pytest tests/test_registry_integration.py -v -m integration
# Run the CLI
python -m cmdforge.cli
# Launch the GUI
cmdforge
# CLI entry points
cmdforge # Main CLI / GUI launcher
cf # Interactive tool picker (fzf-style)
python -m cmdforge.cli # Alternative CLI invocation
```
## Architecture
@@ -76,7 +73,10 @@ Tools are YAML configs with:
### Step Types
1. **Prompt Step**: Calls AI provider with template, stores result in `output_var`. Supports `profile` for AI personas and `strip_fences` for markdown cleanup
1. **Prompt Step**: Calls AI provider with template, stores result in `output_var`
- `profile`: AI persona (system prompt prefix)
- `strip_fences`: Remove markdown code fences from output
- `structured_output`: JSON schema for validated structured responses
2. **Code Step**: Executes Python code via `exec()`, captures specified variables (comma-separated for multiple outputs)
3. **Tool Step**: Calls another tool (meta-tools), supports `args` dict and `provider` override. Dependencies resolved via `resolver.py`
@@ -103,7 +103,7 @@ providers:
command: "echo '[MOCK]'"
```
The `mock` provider is built-in for testing without API calls.
The `mock` provider is built-in for testing without API calls. Use `--provider mock` or `--dry-run` flags when testing tools.
## Web UI & Registry
@@ -145,14 +145,22 @@ CMDFORGE_REGISTRY_DB=/path/to/db PORT=5050 python -m cmdforge.web.app
## Testing Conventions
Tests use `pytest` with common fixtures:
- `temp_tools_dir(tmp_path)`: Redirects `TOOLS_DIR` and `BIN_DIR` to temp directory
- `temp_providers_file(tmp_path)`: Redirects `providers.yaml` to temp directory
Tests use `pytest` without a shared conftest.py. Common patterns:
Mocking strategy:
- **File system**: Use `tmp_path` fixture and patch module-level paths
**Mocking strategy:**
- **File system**: Use `tmp_path` fixture and patch module-level paths (`TOOLS_DIR`, `BIN_DIR`, `PROVIDERS_FILE`)
- **Subprocess calls**: Mock `subprocess.run` and `shutil.which`
- **Provider calls**: Mock `call_provider` to return `ProviderResult`
- **Provider calls**: Mock `call_provider` to return `ProviderResult(text="...", success=True)`
- **Registry calls**: Mock `requests` for API tests
**Example fixture pattern:**
```python
@pytest.fixture
def temp_providers_file(tmp_path):
providers_file = tmp_path / "providers.yaml"
with patch('cmdforge.providers.PROVIDERS_FILE', providers_file):
yield providers_file
```
Integration tests are marked with `@pytest.mark.integration` and require a running registry server.