Add local collections support with GUI, CLI, and publishing workflow

Features:
- Local collection definitions stored in ~/.cmdforge/collections/
- CLI commands: create, show, add, remove, delete, publish, status
- GUI Collections page with local and registry tabs
- Collection publishing with tool resolution and visibility checks
- New API endpoints: GET /api/v1/me, GET /api/v1/tools/.../approved,
  POST /api/v1/collections
- RegistryClient methods: get_me(), has_approved_public_tool(),
  publish_collection()

Implementation:
- collection.py: Collection dataclass, resolve_tool_references(),
  classify_tool_reference(), ToolResolutionResult
- collections_page.py: GUI with background workers for install/publish
- collections_commands.py: Full CLI command implementations
- registry/app.py: New authenticated endpoints with validation

Tests:
- test_collection.py: 27 unit tests for collection module
- test_collection_api.py: 20 tests (8 client, 12 API with Flask skip)

Documentation updated: README, CHANGELOG, CLAUDE.md, tests/README

Co-Authored-By: Claude Opus 4.5 <[email protected]>
This commit is contained in:
rob
2026-01-26 16:50:19 -04:00
co-authored by Claude Opus 4.5
parent 37e74bf339
commit 29869ef59f
14 changed files with 2917 additions and 27 deletions
+27 -1
View File
@@ -2,7 +2,7 @@
## Overview
The test suite covers the core CmdForge modules with **158 unit tests** organized into four main test files:
The test suite covers the core CmdForge modules with **205 unit tests** organized into test files:
| Test File | Module Tested | Test Count | Description |
|-----------|---------------|------------|-------------|
@@ -10,6 +10,8 @@ The test suite covers the core CmdForge modules with **158 unit tests** organize
| `test_runner.py` | `runner.py` | 31 | Variable substitution, step execution |
| `test_providers.py` | `providers.py` | 35 | Provider management, AI calls |
| `test_cli.py` | `cli/` | 20 | CLI commands and arguments |
| `test_collection.py` | `collection.py` | 27 | Collection CRUD, tool resolution |
| `test_collection_api.py` | Registry API | 20 | Collection API endpoints, client methods |
| `test_registry_integration.py` | Registry | 37 | Registry client, resolution (12 require server) |
## Running Tests
@@ -73,6 +75,30 @@ pytest tests/ --cov=cmdforge --cov-report=html
- `TestRefreshCommand` - Regenerate wrapper scripts
- `TestDocsCommand` - View/edit documentation
**test_collection.py** - Collection management
- `TestCollection` - Collection dataclass and CRUD operations
- Create, save, load, delete collections
- Registry URL generation
- YAML serialization roundtrip
- `TestListCollections` - List local collections
- `TestGetCollection` - Get collection by name
- `TestClassifyToolReference` - Classify local vs registry tool refs
- `TestResolveToolReferences` - Transform tool refs for publishing
- Local tool name expansion (name → user/name)
- Visibility issue detection
- Registry tool validation
- Pinned version transformation
- `TestToolResolutionResult` - Resolution result dataclass
**test_collection_api.py** - Collection API and client
- `TestGetMeEndpoint` - GET /api/v1/me endpoint (requires Flask)
- `TestToolApprovedEndpoint` - GET /api/v1/tools/.../approved (requires Flask)
- `TestPostCollectionsEndpoint` - POST /api/v1/collections (requires Flask)
- `TestRegistryClientMethods` - Client methods with mocked requests
- `get_me()` - Get authenticated user info
- `has_approved_public_tool()` - Check tool approval status
- `publish_collection()` - Publish collection to registry
### Integration Tests (require running server)
**test_registry_integration.py** - Registry API tests (marked with `@pytest.mark.integration`)
+298
View File
@@ -0,0 +1,298 @@
"""Tests for collection.py - Collection definitions and management."""
import tempfile
from pathlib import Path
from unittest.mock import patch, MagicMock
import pytest
import yaml
from cmdforge.collection import (
Collection, list_collections, get_collection, COLLECTIONS_DIR,
classify_tool_reference, resolve_tool_references, ToolResolutionResult
)
@pytest.fixture
def temp_collections_dir(tmp_path):
"""Create a temporary collections directory."""
with patch('cmdforge.collection.COLLECTIONS_DIR', tmp_path):
yield tmp_path
class TestCollection:
"""Tests for Collection dataclass."""
def test_create_basic(self):
coll = Collection(name="my-tools", display_name="My Tools")
assert coll.name == "my-tools"
assert coll.display_name == "My Tools"
assert coll.description == ""
assert coll.tools == []
assert coll.pinned == {}
assert coll.tags == []
assert coll.published is False
def test_create_full(self):
coll = Collection(
name="dev-tools",
display_name="Developer Tools",
description="Essential tools for developers",
maintainer="rob",
tools=["summarize", "official/code-review"],
pinned={"official/code-review": "^1.0.0"},
tags=["dev", "productivity"],
published=True,
registry_name="dev-tools"
)
assert coll.name == "dev-tools"
assert coll.maintainer == "rob"
assert len(coll.tools) == 2
assert coll.pinned["official/code-review"] == "^1.0.0"
assert coll.published is True
def test_to_dict(self):
coll = Collection(
name="test-coll",
display_name="Test Collection",
description="Test",
tools=["tool1"],
tags=["test"]
)
d = coll.to_dict()
assert d["name"] == "test-coll"
assert d["display_name"] == "Test Collection"
assert d["tools"] == ["tool1"]
assert d["tags"] == ["test"]
assert d["published"] is False
def test_save_and_load(self, temp_collections_dir):
coll = Collection(
name="my-coll",
display_name="My Collection",
description="A test collection",
tools=["tool1", "owner/tool2"],
pinned={"owner/tool2": "1.0.0"},
tags=["test"]
)
coll.save()
# Verify file exists
path = temp_collections_dir / "my-coll.yaml"
assert path.exists()
# Load and verify
loaded = Collection.load("my-coll")
assert loaded is not None
assert loaded.name == "my-coll"
assert loaded.display_name == "My Collection"
assert loaded.tools == ["tool1", "owner/tool2"]
assert loaded.pinned == {"owner/tool2": "1.0.0"}
def test_load_nonexistent(self, temp_collections_dir):
loaded = Collection.load("nonexistent")
assert loaded is None
def test_load_empty_yaml(self, temp_collections_dir):
# Create empty YAML file
path = temp_collections_dir / "empty.yaml"
path.write_text("")
loaded = Collection.load("empty")
assert loaded is not None
assert loaded.name == "empty"
def test_delete(self, temp_collections_dir):
coll = Collection(name="to-delete", display_name="To Delete")
coll.save()
path = temp_collections_dir / "to-delete.yaml"
assert path.exists()
coll.delete()
assert not path.exists()
def test_get_registry_url(self):
with patch('cmdforge.collection.get_registry_url', return_value="https://cmdforge.brrd.tech/api/v1"):
coll = Collection(name="my-coll", display_name="My Coll")
url = coll.get_registry_url()
assert url == "https://cmdforge.brrd.tech/collections/my-coll"
def test_get_registry_url_custom_base(self):
with patch('cmdforge.collection.get_registry_url', return_value="https://custom.example.com/api/v1"):
coll = Collection(name="my-coll", display_name="My Coll")
url = coll.get_registry_url()
assert url == "https://custom.example.com/collections/my-coll"
def test_get_registry_url_no_api_suffix(self):
with patch('cmdforge.collection.get_registry_url', return_value="https://cmdforge.brrd.tech"):
coll = Collection(name="my-coll", display_name="My Coll")
url = coll.get_registry_url()
assert url == "https://cmdforge.brrd.tech/collections/my-coll"
def test_get_registry_url_uses_registry_name(self):
with patch('cmdforge.collection.get_registry_url', return_value="https://cmdforge.brrd.tech/api/v1"):
coll = Collection(name="local-name", display_name="My Coll", registry_name="published-name")
url = coll.get_registry_url()
assert url == "https://cmdforge.brrd.tech/collections/published-name"
class TestListCollections:
"""Tests for list_collections function."""
def test_empty_dir(self, temp_collections_dir):
result = list_collections()
assert result == []
def test_with_collections(self, temp_collections_dir):
# Create some collections
Collection(name="alpha", display_name="Alpha").save()
Collection(name="beta", display_name="Beta").save()
Collection(name="gamma", display_name="Gamma").save()
result = list_collections()
assert result == ["alpha", "beta", "gamma"] # Sorted
def test_ignores_non_yaml_files(self, temp_collections_dir):
Collection(name="valid", display_name="Valid").save()
(temp_collections_dir / "not-yaml.txt").write_text("ignored")
result = list_collections()
assert result == ["valid"]
class TestGetCollection:
"""Tests for get_collection function."""
def test_existing(self, temp_collections_dir):
Collection(name="test", display_name="Test", description="Desc").save()
coll = get_collection("test")
assert coll is not None
assert coll.name == "test"
assert coll.description == "Desc"
def test_nonexistent(self, temp_collections_dir):
coll = get_collection("nonexistent")
assert coll is None
class TestClassifyToolReference:
"""Tests for classify_tool_reference function."""
def test_local_tool(self):
owner, name, is_local = classify_tool_reference("summarize")
assert owner == ""
assert name == "summarize"
assert is_local is True
def test_registry_tool(self):
owner, name, is_local = classify_tool_reference("official/summarize")
assert owner == "official"
assert name == "summarize"
assert is_local is False
def test_registry_tool_nested_name(self):
owner, name, is_local = classify_tool_reference("user/my-tool")
assert owner == "user"
assert name == "my-tool"
assert is_local is False
class TestResolveToolReferences:
"""Tests for resolve_tool_references function."""
@pytest.fixture
def mock_client(self):
client = MagicMock()
client.get_my_tool_status.side_effect = lambda name: {"status": "approved"}
client.has_approved_public_tool.return_value = True
return client
def test_all_registry_tools(self, mock_client):
tools = ["official/tool1", "user/tool2"]
pinned = {"official/tool1": "^1.0.0"}
result = resolve_tool_references(tools, pinned, "myuser", mock_client)
assert result.registry_refs == ["official/tool1", "user/tool2"]
assert result.transformed_pinned == {"official/tool1": "^1.0.0"}
assert result.local_unpublished == []
def test_local_tool_not_published(self, mock_client):
from cmdforge.registry_client import RegistryError
mock_client.get_my_tool_status.side_effect = RegistryError(
code="TOOL_NOT_FOUND", message="Not found"
)
with patch('cmdforge.collection._get_tool_visibility_from_yaml', return_value='public'):
tools = ["my-local-tool"]
result = resolve_tool_references(tools, {}, "myuser", mock_client)
assert "myuser/my-local-tool" in result.registry_refs
assert "my-local-tool" in result.local_unpublished
def test_local_tool_published_approved(self, mock_client):
mock_client.get_my_tool_status.return_value = {"status": "approved"}
with patch('cmdforge.collection._get_tool_visibility_from_yaml', return_value='public'):
tools = ["my-tool"]
result = resolve_tool_references(tools, {}, "myuser", mock_client)
assert "myuser/my-tool" in result.registry_refs
assert result.local_unpublished == []
assert ("my-tool", "approved", True) in result.local_published
def test_transforms_pinned_for_local_tools(self, mock_client):
mock_client.get_my_tool_status.return_value = {"status": "approved"}
with patch('cmdforge.collection._get_tool_visibility_from_yaml', return_value='public'):
tools = ["my-tool"]
pinned = {"my-tool": "1.0.0"}
result = resolve_tool_references(tools, pinned, "myuser", mock_client)
assert "myuser/my-tool" in result.transformed_pinned
assert result.transformed_pinned["myuser/my-tool"] == "1.0.0"
def test_visibility_issues_detected(self, mock_client):
with patch('cmdforge.collection._get_tool_visibility_from_yaml', return_value='private'):
tools = ["private-tool"]
result = resolve_tool_references(tools, {}, "myuser", mock_client)
assert ("private-tool", "private") in result.visibility_issues
def test_registry_tool_not_approved(self, mock_client):
mock_client.has_approved_public_tool.return_value = False
tools = ["official/unapproved"]
result = resolve_tool_references(tools, {}, "myuser", mock_client)
assert ("official/unapproved", "no approved public version") in result.registry_tool_issues
class TestToolResolutionResult:
"""Tests for ToolResolutionResult dataclass."""
def test_empty_result(self):
result = ToolResolutionResult(
registry_refs=[],
transformed_pinned={},
local_unpublished=[],
local_published=[],
visibility_issues=[],
registry_tool_issues=[]
)
assert len(result.registry_refs) == 0
assert len(result.local_unpublished) == 0
def test_with_data(self):
result = ToolResolutionResult(
registry_refs=["user/tool1", "user/tool2"],
transformed_pinned={"user/tool1": "^1.0"},
local_unpublished=["tool3"],
local_published=[("tool1", "approved", True)],
visibility_issues=[("tool4", "private")],
registry_tool_issues=[("other/tool", "not found")]
)
assert len(result.registry_refs) == 2
assert len(result.local_unpublished) == 1
assert len(result.visibility_issues) == 1
+365
View File
@@ -0,0 +1,365 @@
"""Tests for collection-related API endpoints.
Tests for:
- GET /api/v1/me
- GET /api/v1/tools/<owner>/<name>/approved
- POST /api/v1/collections
"""
import json
import os
import tempfile
from pathlib import Path
from unittest.mock import patch, MagicMock
import pytest
# Check if Flask is available for API tests
try:
import flask
HAS_FLASK = True
except ImportError:
HAS_FLASK = False
flask_required = pytest.mark.skipif(not HAS_FLASK, reason="Flask not installed")
@pytest.fixture
def app():
"""Create Flask test app with in-memory database."""
pytest.importorskip("flask", reason="Flask not installed")
from cmdforge.registry.app import create_app
with tempfile.NamedTemporaryFile(suffix='.db', delete=False) as f:
db_path = f.name
with patch.dict(os.environ, {"CMDFORGE_REGISTRY_DB": db_path}):
app = create_app()
app.config["TESTING"] = True
yield app
# Cleanup
Path(db_path).unlink(missing_ok=True)
@pytest.fixture
def client(app):
"""Create test client."""
return app.test_client()
@pytest.fixture
def auth_headers(app):
"""Create auth headers with a valid token."""
import hashlib
from datetime import datetime
from cmdforge.registry.db import connect_db
token = "test-token"
token_hash = hashlib.sha256(token.encode()).hexdigest()
conn = connect_db()
try:
row = conn.execute(
"SELECT id FROM publishers WHERE slug = ?",
["testuser"],
).fetchone()
if row:
publisher_id = row["id"]
else:
conn.execute(
"""
INSERT INTO publishers (email, password_hash, slug, display_name, role)
VALUES (?, ?, ?, ?, ?)
""",
["[email protected]", "x", "testuser", "Test User", "user"],
)
publisher_id = conn.execute(
"SELECT id FROM publishers WHERE slug = ?",
["testuser"],
).fetchone()["id"]
# Insert token if missing
token_row = conn.execute(
"SELECT id FROM api_tokens WHERE token_hash = ?",
[token_hash],
).fetchone()
if not token_row:
conn.execute(
"""
INSERT INTO api_tokens (publisher_id, token_hash, name, created_at)
VALUES (?, ?, ?, ?)
""",
[publisher_id, token_hash, "test-token", datetime.utcnow().isoformat()],
)
# Insert a public approved tool for tests if missing
tool_row = conn.execute(
"SELECT id FROM tools WHERE owner = ? AND name = ?",
["testuser", "tool1"],
).fetchone()
if not tool_row:
conn.execute(
"""
INSERT INTO tools (owner, name, version, description, category, tags,
config_yaml, readme, publisher_id, visibility, moderation_status, published_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
"testuser", "tool1", "1.0.0", "Test tool", "Other", "[]",
"name: tool1\nversion: 1.0.0\n", "", publisher_id,
"public", "approved", datetime.utcnow().isoformat(),
],
)
conn.commit()
finally:
conn.close()
return {"Authorization": f"Bearer {token}"}
@flask_required
class TestGetMeEndpoint:
"""Tests for GET /api/v1/me endpoint."""
def test_requires_auth(self, client):
response = client.get('/api/v1/me')
assert response.status_code == 401
def test_returns_user_info(self, client, auth_headers):
response = client.get('/api/v1/me', headers=auth_headers)
assert response.status_code == 200
data = response.get_json()
assert data["data"]["slug"] == "testuser"
@flask_required
class TestToolApprovedEndpoint:
"""Tests for GET /api/v1/tools/<owner>/<name>/approved endpoint."""
def test_invalid_owner_format(self, client):
response = client.get('/api/v1/tools/invalid@owner/tool/approved')
assert response.status_code == 400
def test_invalid_name_format(self, client):
response = client.get('/api/v1/tools/valid/invalid@name/approved')
assert response.status_code == 400
def test_tool_not_found(self, client):
response = client.get('/api/v1/tools/nonexistent/tool/approved')
data = json.loads(response.data)
# Should return success but with has_approved_public_version = False
assert response.status_code == 200
assert data['data']['has_approved_public_version'] is False
@flask_required
class TestPostCollectionsEndpoint:
"""Tests for POST /api/v1/collections endpoint."""
def test_requires_auth(self, client):
response = client.post('/api/v1/collections', json={
"name": "test-coll",
"display_name": "Test Collection",
"tools": ["official/tool1"]
})
assert response.status_code == 401
def test_invalid_name_format(self, client, auth_headers):
response = client.post('/api/v1/collections', headers=auth_headers, json={
"name": "Invalid Name",
"display_name": "Test Collection",
"tools": ["testuser/tool1"]
})
assert response.status_code == 400
def test_missing_display_name(self, client, auth_headers):
response = client.post('/api/v1/collections', headers=auth_headers, json={
"name": "test-coll",
"tools": ["testuser/tool1"]
})
assert response.status_code == 400
def test_missing_tools(self, client, auth_headers):
response = client.post('/api/v1/collections', headers=auth_headers, json={
"name": "test-coll",
"display_name": "Test Collection",
})
assert response.status_code == 400
def test_tools_must_be_list(self, client, auth_headers):
response = client.post('/api/v1/collections', headers=auth_headers, json={
"name": "test-coll",
"display_name": "Test Collection",
"tools": "not-a-list"
})
assert response.status_code == 400
def test_pinned_must_be_dict(self, client, auth_headers):
response = client.post('/api/v1/collections', headers=auth_headers, json={
"name": "test-coll",
"display_name": "Test Collection",
"tools": ["testuser/tool1"],
"pinned": ["not", "a", "dict"]
})
assert response.status_code == 400
def test_tags_must_be_list(self, client, auth_headers):
response = client.post('/api/v1/collections', headers=auth_headers, json={
"name": "test-coll",
"display_name": "Test Collection",
"tools": ["testuser/tool1"],
"tags": "not-a-list"
})
assert response.status_code == 400
class TestRegistryClientMethods:
"""Tests for new RegistryClient methods."""
@pytest.fixture
def mock_session(self):
"""Create a mock requests session."""
session = MagicMock()
return session
def test_get_me(self, mock_session):
from cmdforge.registry_client import RegistryClient
client = RegistryClient(token="test-token")
client._session = mock_session
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"data": {"id": 1, "slug": "testuser", "role": "user"}
}
mock_session.request.return_value = mock_response
result = client.get_me()
assert result["slug"] == "testuser"
assert result["role"] == "user"
def test_get_me_unauthorized(self, mock_session):
from cmdforge.registry_client import RegistryClient, RegistryError
client = RegistryClient(token="bad-token")
client._session = mock_session
mock_response = MagicMock()
mock_response.status_code = 401
mock_response.json.return_value = {
"error": {"code": "UNAUTHORIZED", "message": "Invalid token"}
}
mock_session.request.return_value = mock_response
with pytest.raises(RegistryError) as exc_info:
client.get_me()
assert exc_info.value.code == "UNAUTHORIZED"
def test_has_approved_public_tool_true(self, mock_session):
from cmdforge.registry_client import RegistryClient
client = RegistryClient()
client._session = mock_session
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"data": {"has_approved_public_version": True}
}
mock_session.request.return_value = mock_response
result = client.has_approved_public_tool("official", "summarize")
assert result is True
def test_has_approved_public_tool_false(self, mock_session):
from cmdforge.registry_client import RegistryClient
client = RegistryClient()
client._session = mock_session
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"data": {"has_approved_public_version": False}
}
mock_session.request.return_value = mock_response
result = client.has_approved_public_tool("official", "pending-tool")
assert result is False
def test_has_approved_public_tool_not_found(self, mock_session):
from cmdforge.registry_client import RegistryClient, RegistryError
client = RegistryClient()
client._session = mock_session
mock_response = MagicMock()
mock_response.status_code = 404
mock_session.request.return_value = mock_response
with pytest.raises(RegistryError) as exc_info:
client.has_approved_public_tool("nonexistent", "tool")
assert exc_info.value.code == "TOOL_NOT_FOUND"
def test_publish_collection_success(self, mock_session):
from cmdforge.registry_client import RegistryClient
client = RegistryClient(token="test-token")
client._session = mock_session
mock_response = MagicMock()
mock_response.status_code = 201
mock_response.json.return_value = {
"data": {"name": "my-coll", "display_name": "My Collection", "created": True}
}
mock_session.request.return_value = mock_response
result = client.publish_collection({
"name": "my-coll",
"display_name": "My Collection",
"tools": ["official/tool1"]
})
assert result["name"] == "my-coll"
assert result["created"] is True
def test_publish_collection_conflict(self, mock_session):
from cmdforge.registry_client import RegistryClient, RegistryError
client = RegistryClient(token="test-token")
client._session = mock_session
mock_response = MagicMock()
mock_response.status_code = 409
mock_session.request.return_value = mock_response
with pytest.raises(RegistryError) as exc_info:
client.publish_collection({
"name": "existing-coll",
"display_name": "Existing",
"tools": ["official/tool1"]
})
assert exc_info.value.code == "COLLECTION_EXISTS"
def test_publish_collection_update(self, mock_session):
from cmdforge.registry_client import RegistryClient
client = RegistryClient(token="test-token")
client._session = mock_session
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"data": {"name": "my-coll", "display_name": "My Collection Updated", "updated": True}
}
mock_session.request.return_value = mock_response
result = client.publish_collection({
"name": "my-coll",
"display_name": "My Collection Updated",
"tools": ["official/tool1", "official/tool2"]
})
assert result["updated"] is True