Commit Graph

12 Commits

Author SHA1 Message Date
rob 5159450feb Add semantic search (RAG) for registry tool discovery
Users can now find tools by describing what they need in natural language.
Uses Ollama embeddings (nomic-embed-text) on AI-Server for vector similarity
search. Available via CLI (registry describe), GUI (AI search row), and API.

New files:
- registry/embeddings.py: core embedding logic (Ollama API, cosine similarity,
  pack/unpack vectors, backfill)
- tests/test_embeddings.py: 16 unit tests

Modified:
- registry/db.py: tool_embeddings table (schema + migration)
- registry/settings.py: embeddings.* settings (ollama_url, model, enabled, min_score)
- registry/app.py: semantic-search endpoint, publish hook, admin backfill/status
- registry_client.py: semantic_search() method with error surfacing
- gui/pages/registry_page.py: AI search row with SemanticSearchWorker
- cli/__init__.py + registry_commands.py: registry describe subcommand

Backfill required after deploy: POST /api/v1/admin/embeddings/backfill

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 01:52:27 -04:00
rob 0ee21f27f7 Add admin publish-as-official, hash-based status sync, and GUI fixes
Registry:
- Add admin owner override for publishing tools as "official"
- Add POST /api/v1/tools/status-by-hash batch endpoint for status lookup
  scoped to publisher_id (works for tools published under any owner)

GUI:
- Add "Publish as" dropdown in publish dialog for admin users
- Add "installed" tool state (teal with arrow indicator) for registry-installed tools
- Fix tool editing for official/* qualified tool names (_get_qualified_name helper)
- Fix cancel navigation returning to wrong page (Welcome instead of Tools)
- Fix collections tab not refreshing after publish
- Refactor StatusSyncWorker to use batch hash lookup (1 request instead of N)
  with chunking (100 max) and hash collision handling

CLI:
- Switch registry status sync to hash-based lookup
- Add collection dependency checking and unpublished dep detection

Publish dialog cleanup:
- Move yaml import to module level, remove duplicate _bump_patch_version
- Fix owner combo using currentText() for reliable selection

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 17:18:37 -04:00
rob 071ade0ffb Add system dependencies support for tools
Tools can now declare system-level package dependencies (apt, brew, pacman, dnf)
that get checked and optionally installed when the tool is installed or run.

Features:
- SystemDependency dataclass with short form (string) and long form (dict)
- New system_deps.py module with platform detection and installation
- `cmdforge system-deps <tool>` command to check/install system packages
- `cmdforge check` now shows both CmdForge and system dependencies
- `cmdforge registry install` prompts for system dep installation
- GUI: System Dependencies section in Tool Builder with add/edit dialog
- Runner warns about missing system deps before execution
- Integration with project install (manifest and lock-based)

Also includes:
- Quote paths in wrapper scripts for spaces support
- Tests for value type preservation in code steps
- Unskip invalid tool name test

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 16:31:37 -04:00
rob 8eff55ed1e Add tool settings files for configurable tool defaults
Tools can now ship with defaults.yaml containing configurable settings
that users can customize via settings.yaml (auto-created from defaults).

Features:
- ensure_settings() helper copies defaults to settings on first use
- Settings available as {settings.key} in templates (scalars only)
- Full dict access via settings['key'] in code steps
- CLI: cmdforge settings <tool> show/edit/reset/diff
- GUI: Defaults editor in Tool Builder, Configure button on Tools page
- Registry: defaults.yaml published with tools, included in downloads
- Secret detection warning on publish (api_key, password, token, etc.)

Fully backward compatible - tools without defaults work unchanged.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 11:56:07 -04:00
rob 3e7eaeb504 Add transitive dependency resolution and lock file support
Implements two major features for project dependency management:

Transitive Dependency Resolution:
- DependencyGraph/DependencyNode for structured dependency tracking
- DependencyGraphBuilder with recursive resolution and cycle detection
- Topological sorting for correct install order (dependencies first)
- Version conflict detection with provenance tracking
- Full semver support: ^, ~, >=, <=, >, <, = constraints
- New `cmdforge deps tree` command to visualize dependency tree
- Install flags: --dry-run, --force, --verbose

Lock File Support (cmdforge.lock):
- Lockfile/LockedPackage/LockfileMetadata dataclasses
- Records exact versions of all dependencies (direct + transitive)
- Integrity verification via SHA256 hashes
- `cmdforge lock` to generate/update lock file
- `cmdforge verify` to check installed tools match lock
- Install flags: --frozen, --strict-frozen, --ignore-lock

62 new tests for version, dependency_graph, and lockfile modules.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 22:05:09 -04:00
rob 29869ef59f Add local collections support with GUI, CLI, and publishing workflow
Features:
- Local collection definitions stored in ~/.cmdforge/collections/
- CLI commands: create, show, add, remove, delete, publish, status
- GUI Collections page with local and registry tabs
- Collection publishing with tool resolution and visibility checks
- New API endpoints: GET /api/v1/me, GET /api/v1/tools/.../approved,
  POST /api/v1/collections
- RegistryClient methods: get_me(), has_approved_public_tool(),
  publish_collection()

Implementation:
- collection.py: Collection dataclass, resolve_tool_references(),
  classify_tool_reference(), ToolResolutionResult
- collections_page.py: GUI with background workers for install/publish
- collections_commands.py: Full CLI command implementations
- registry/app.py: New authenticated endpoints with validation

Tests:
- test_collection.py: 27 unit tests for collection module
- test_collection_api.py: 20 tests (8 client, 12 API with Flask skip)

Documentation updated: README, CHANGELOG, CLAUDE.md, tests/README

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 16:50:19 -04:00
rob e18a575f76 Add TUI publish feature and fix web UI dropdown
- Add Publish button to TUI for publishing tools directly from the
  graphical interface without using the command line
- Fix user dropdown menu in web UI header that stayed open after
  clicking (replaced Alpine.js directives with vanilla JS)
- Update test to use correct registry URL (cmdforge.brrd.tech)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-13 21:12:42 -04:00
rob 497fe87fc5 Rename project from SmartTools to CmdForge
Major rename operation:
- Rename Python package: smarttools -> cmdforge
- Update CLI entry point: smarttools -> cmdforge
- Update all imports and module references
- Update pyproject.toml with new name, URLs, entry point
- Update all HTML templates with new branding
- Update documentation (CLAUDE.md, README.md, docs/*, wiki/*)
- Update environment variables:
  - SMARTTOOLS_ENV -> CMDFORGE_ENV
  - SMARTTOOLS_REGISTRY_DB -> CMDFORGE_REGISTRY_DB
  - SMARTTOOLS_TOKEN -> CMDFORGE_TOKEN
  - SMARTTOOLS_REGISTRY_* -> CMDFORGE_REGISTRY_*
- Update Dockerfile and docker-compose.yml
- Update tests to use new package name
- Update scripts and examples
- Update package.json and tailwind.config.js

All 158 unit tests pass. The CLI is working correctly with the
new cmdforge command.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 22:15:55 -04:00
rob 02842e4287 Include all dependencies and tests in Docker image
- Add tests/ to Docker image for in-container testing
- Install [all,dev] dependencies (includes Flask, registry, TUI)
- Make integration tests configurable via REGISTRY_URL env var
- Add error handling to publish test fixture for rate limiting

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 03:51:46 -04:00
rob cdb17db43a Add comprehensive test suite for core modules
New test files:
- test_tool.py: 35 tests for Tool, ToolArgument, steps, persistence
- test_runner.py: 31 tests for variable substitution, step execution
- test_providers.py: 35 tests for provider management, mocked calls
- test_cli.py: 20 tests for CLI commands
- tests/README.md: Test suite documentation

Removed placeholder test.py (triangle calculator).

Total: 158 unit tests passing, 12 integration tests (require server)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 03:07:16 -04:00
rob 34428f0e72 Add SmartTools Registry and Web UI (Phases 1-7)
Registry Features (Phases 1-6):
- Tool manifest format and validation
- Registry client for API communication
- Resolver integration for tool installation
- CLI commands: install, search, browse, publish, auth
- TUI browse interface with urwid
- Smart features: auto-categorization, similarity suggestions

Web UI (Phase 7):
- Flask blueprint with server-side sessions
- Authentication routes (login, register, logout)
- Public pages: landing, tools browse, tool detail, search
- Dashboard: overview, tools management, API tokens, settings
- Template components: cards, forms, callouts, code blocks
- Tailwind CSS build pipeline
- SEO infrastructure (sitemap, robots.txt)
- Cookie consent and privacy compliance

Infrastructure:
- SQLite database with FTS5 search
- Rate limiting for API endpoints
- Git-based tool submission workflow
- Session management with auto-cleanup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 19:03:41 -04:00
rob 65279bd8c8 Add Docker support and documentation updates
- Add Dockerfile for containerized builds
- Add .dockerignore for cleaner builds
- Add CLAUDE.md for development guidance
- Update README and docs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-23 23:40:13 -04:00