diff --git a/docs/AUTOMATION.md b/docs/AUTOMATION.md deleted file mode 100644 index 55a397e..0000000 --- a/docs/AUTOMATION.md +++ /dev/null @@ -1,623 +0,0 @@ -# CascadingDev Automation System - -## Overview - -The CascadingDev automation system processes discussion files during git commits to maintain structured summaries. It operates in two phases: - -**Phase 1 (Basic)**: Always runs, no dependencies -- Parses `VOTE:` lines from discussions -- Tracks latest vote per participant -- Updates summary `.sum.md` files automatically - -**Phase 2 (AI-Enhanced)**: Optional, requires Claude API -- Extracts questions, action items, and decisions using AI -- Tracks @mentions and awaiting replies -- Processes only incremental changes (git diff) -- Maintains timeline and structured summaries - -## Architecture - -``` -automation/ -├── workflow.py # Main orchestrator, called by pre-commit hook -├── agents.py # Claude-powered extraction agents -├── summary.py # Summary file formatter and updater -└── __init__.py -``` - -## Phase 1: Vote Tracking (Always Enabled) - -### How It Works - -1. **Pre-commit hook** triggers `automation/workflow.py --status` -2. Finds all staged `.discussion.md` files -3. Parses entire file for `VOTE:` lines -4. Maintains latest vote per participant -5. Updates corresponding `.sum.md` file's VOTES section -6. Auto-stages updated summaries - -### Vote Format - -```markdown -- ParticipantName: Any comment text. VOTE: READY|CHANGES|REJECT -``` - -**Rules:** -- Only the **latest** vote per participant counts -- Case-insensitive vote tokens (vote:, VOTE:, Vote:) -- Three valid values: READY, CHANGES, REJECT -- Must follow participant bullet format: `- Name: ...` - -### Example - -**Discussion file:** -```markdown -- Alice: I like this approach. VOTE: READY -- Bob: We need tests first. VOTE: CHANGES -- Alice: Good point, updated the plan. VOTE: CHANGES -``` - -**Resulting summary:** -```markdown - -## Votes (latest per participant) -READY: 0 • CHANGES: 2 • REJECT: 0 -- Alice: CHANGES -- Bob: CHANGES - -``` - -## Phase 2: AI-Enhanced Processing (Optional) - -### Requirements - -Phase 2 supports multiple AI providers with automatic fallback chains and intelligent model selection. Configuration is managed through `config/ai.yml` (copied into every generated project). - -### Configuration: config/ai.yml (Primary Method) - -**Edit the configuration file** to set your preferred AI providers and fallback chain: - -```yaml -# config/ai.yml -runner: - # Default command chain (balanced speed/quality) - command_chain: - - "claude -p" - - "codex --model gpt-5" - - "gemini --model gemini-2.5-flash" - - # Fast command chain (simple tasks: vote counting, gate checks) - # Used when model_hint: fast in .ai-rules.yml - command_chain_fast: - - "claude -p" # Auto-selects Haiku via subagent - - "codex --model gpt-5-mini" - - "gemini --model gemini-2.5-flash" - - # Quality command chain (complex tasks: design, implementation planning) - # Used when model_hint: quality in .ai-rules.yml - command_chain_quality: - - "claude -p" # Auto-selects Sonnet via subagent - - "codex --model o3" - - "gemini --model gemini-2.5-pro" - - sentinel: "CASCADINGDEV_NO_CHANGES" -``` - -### How it works: -- The automation runner (`patcher.py`) iterates through the `command_chain` from top to bottom. -- It attempts to generate a patch or summary with the first provider. -- If the provider fails (e.g., API error, invalid output), it automatically retries with the next provider in the chain. -- This continues until a provider succeeds or the chain is exhausted. -- This resilience ensures that the automation can proceed even if one AI provider is temporarily unavailable or performs poorly on a given task. -- `.ai-rules.yml` can specify `model_hint: fast` or `model_hint: quality` per rule -- Fast models (Haiku, GPT-5-mini) handle simple tasks (vote counting, gate checks) -- Quality models (Sonnet, O3) handle complex tasks (design discussions, planning) -- This optimization reduces costs by ~70% while maintaining quality - -### Environment Overrides (Optional) - -**Temporarily override** `config/ai.yml` for a single commit: - -```bash -# Override command for this commit only -CDEV_AI_COMMAND="claude -p" git commit -m "message" - -# Chain multiple providers with || delimiter -CDEV_AI_COMMAND="claude -p || codex --model gpt-5" git commit -m "message" -``` - -Environment variables take precedence but don't modify the config file. - -Common non-interactive setups: - -| Provider | CLI Tool | Command Example | Authentication | Notes | -| --- | --- | --- | --- | --- | -| Claude | `claude` | `claude -p` | Run `claude` and follow prompts to sign in | Supports custom subagents in `~/.claude/agents/`. Create with `./tools/setup_claude_agents.sh`. Uses Haiku (fast) or Sonnet (quality). | -| OpenAI | `codex` | `codex --model gpt-5` | Run `codex` and sign in with ChatGPT account | Codex CLI is OpenAI's terminal coding agent. Default model: GPT-5. Use `gpt-5-mini` for faster, cheaper responses. | -| Google | `gemini` | `gemini --model gemini-2.5-flash` | Run `gemini` and sign in with Google account | Free tier: 60 req/min. Use `gemini-2.5-flash` (fast) or `gemini-2.5-pro` (1M context, quality). Open source (Apache 2.0). | - -**Recommended Setup:** Use the provided setup script to create Claude subagents: -```bash -# One-time setup (creates ~/.claude/agents/cdev-patch.md and cdev-patch-quality.md) -./tools/setup_claude_agents.sh -``` - -### Automated Discussion Status Promotion - -A key feature of the automation system is its ability to automatically update the status of a discussion file based on participant votes. This moves a feature, design, or other discussion through its lifecycle without manual intervention. - -**How it works:** - -1. The `workflow.py` script is triggered on commit. -2. It scans staged discussion files for YAML front matter containing a `promotion_rule`. -3. It tallies `VOTE: READY` and `VOTE: REJECT` votes from eligible participants. -4. If the vote counts meet the configured thresholds, the script updates the `status:` field in the file's front matter (e.g., from `OPEN` to `READY_FOR_DESIGN`). -5. The updated discussion file is automatically staged. - -**Configuration (in discussion file front matter):** - -The entire process is controlled by a `promotion_rule` block within the YAML front matter of a discussion file (e.g., `feature.discussion.md`). - -```yaml ---- -type: feature-discussion -stage: feature -status: OPEN -promotion_rule: - allow_agent_votes: false - ready_min_eligible_votes: 2 - reject_min_eligible_votes: 1 - ready_status: "READY_FOR_DESIGN" - reject_status: "FEATURE_REJECTED" ---- - -Alice: Looks good. VOTE: READY -Bob: I agree. VOTE: READY -``` - -**Parameters:** - -* `allow_agent_votes` (boolean, optional): If `true`, votes from participants named `AI_*` will be counted. Defaults to `false`. -* `ready_min_eligible_votes` (integer | "all", optional): The number of `READY` votes required to promote the status. Can be an integer or the string `"all"` (requiring all eligible voters to vote `READY`). Defaults to `2`. -* `reject_min_eligible_votes` (integer | "all", optional): The number of `REJECT` votes required to reject the discussion. Defaults to `1`. -* `ready_status` (string, optional): The target status to set when the `ready` threshold is met. Defaults to stage-specific values (e.g., `READY_FOR_DESIGN` for the `feature` stage). -* `reject_status` (string, optional): The target status to set when the `reject` threshold is met. Defaults to stage-specific values (e.g., `FEATURE_REJECTED`). - -This automation turns the discussion and voting process into a state machine, enabling a self-driving project workflow. - - -This creates two subagent files: -- `cdev-patch.md` - Uses Haiku model (fast, cost-efficient) - auto-selected when TASK COMPLEXITY: FAST -- `cdev-patch-quality.md` - Uses Sonnet model (higher quality) - auto-selected when TASK COMPLEXITY: QUALITY - -The `claude -p` command automatically selects the appropriate subagent based on the `TASK COMPLEXITY` hint in the prompt: -- Simple tasks (vote counting, gate checks) → `command_chain_fast` → Haiku -- Complex tasks (design, implementation planning) → `command_chain_quality` → Sonnet -- Default tasks → `command_chain` → auto-select based on complexity - -**Option 2: Direct API (Alternative)** -```bash -pip install anthropic -export ANTHROPIC_API_KEY="sk-ant-..." -``` - -If no AI provider is configured or responding, Phase 2 features are silently skipped and only Phase 1 (votes) runs. - -### Features - -#### 1. @Mention Tracking - -Extracts `@Name` and `@all` mentions from discussions to track who's waiting for replies. - -**Format:** -```markdown -- Alice: @Bob what do you think about OAuth2? -- Carol: @all please review by Friday -``` - -**Summary section:** -```markdown - -## Awaiting Replies - -### @Bob -- @Alice: What do you think about OAuth2? - -### @all -- @Carol: Please review by Friday - -``` - -#### 2. Question Tracking - -Identifies questions and tracks their answers. - -**Markers (optional but recommended):** -```markdown -- Alice: Q: Should we use OAuth2 or JWT? -- Bob: A: I'd recommend OAuth2 for third-party auth. -``` - -**Also detects:** -- Lines ending with `?` -- `Question:` prefix -- `Re:` replies (indicate partial answers) - -**Summary section:** -```markdown - -## Open Questions -- @Alice: Should we cache API responses? - -### Partially Answered: -- @Bob: What about rate limiting? - - Partial answer: We'll use token bucket algorithm - -``` - -#### 3. Action Item Management - -Tracks tasks from creation → assignment → completion. - -**Markers (optional but recommended):** -```markdown -- Alice: TODO: Research OAuth2 libraries -- Bob: I'll handle the JWT implementation. -- Alice: DONE: Completed library research, recommending authlib. -- Dave: ACTION: Review security implications -``` - -**Summary section:** -```markdown - -## Action Items - -### TODO (unassigned): -- [ ] Document the authentication flow (suggested by @Carol) - -### In Progress: -- [ ] Implement JWT token validation (@Bob) - -### Completed: -- [x] Research OAuth2 libraries (@Alice) - -``` - -#### 4. Decision Logging (ADR-Style) - -Captures architectural decisions with rationale. - -**Markers (optional but recommended):** -```markdown -- Alice: DECISION: Use OAuth2 + JWT hybrid approach. - Rationale: OAuth2 for robust third-party auth, JWT for stateless sessions. -``` - -**Also detects:** -- "We decided to..." -- "Going with X because..." -- Vote consensus (multiple READY votes) - -**Summary section:** -```markdown - -## Decisions (ADR-style) - -### Decision 1: Use OAuth2 + JWT hybrid approach -- **Proposed by:** @Alice -- **Supported by:** @Bob, @Carol -- **Rationale:** OAuth2 for robust third-party auth, JWT for stateless sessions -- **Alternatives considered:** - - Pure JWT authentication - - Session-based auth with cookies - -``` - -## Conversation Guidelines - -### Natural Conversation (Recommended) - -**Write naturally - AI normalization extracts markers automatically:** - -```markdown -# Examples of natural conversation that AI understands: - -- Alice: I think we should use OAuth2. Does anyone know if we need OAuth 2.1 specifically? - VOTE: READY - -- Bob: Good question Alice. I'm making a decision here - we'll use OAuth 2.0 for now. - @Carol can you research migration paths to 2.1? VOTE: CHANGES - -- Carol: I've completed the OAuth research. We can upgrade later without breaking changes. - VOTE: READY -``` - -**AI normalization (via `agents.py`) extracts:** -- Decisions from natural language ("I'm making a decision here - ...") -- Questions from conversational text ("Does anyone know if...") -- Action items with @mentions ("@Carol can you research...") -- Votes (always tracked: `VOTE: READY|CHANGES|REJECT`) - -### Explicit Markers (Fallback) - -**If AI is unavailable, these explicit line-start markers work as fallback:** - -```markdown -# Markers (✅ = works without AI as simple fallback) - -QUESTION: # ✅ Explicit question marker -Q: # ✅ Short form - -TODO: # ✅ New unassigned task -ACTION: # ✅ Task with implied ownership -ASSIGNED: @name # ✅ Claimed task -DONE: # ✅ Mark task complete - -DECISION: # ✅ Architectural decision - -VOTE: READY|CHANGES|REJECT # ✅ ALWAYS tracked (with or without AI) - -@Name # ✅ Mention extraction (simple regex) -``` - -**Example with explicit markers:** -```markdown -- Alice: QUESTION: Should we support OAuth2? -- Bob: TODO: Research OAuth2 libraries -- Bob: ASSIGNED: OAuth2 library research -- Carol: DECISION: Use OAuth2 for authentication -- Dave: @all Please review. VOTE: READY -``` - -### Two-Tier Architecture - -1. **AI Normalization (Primary):** Handles natural conversation, embedded markers, context understanding -2. **Simple Fallback:** Handles explicit line-start markers when AI unavailable - -Benefits: -- ✅ Participants write naturally without strict formatting -- ✅ Resilient (multi-provider fallback: claude → codex → gemini) -- ✅ Works offline/API-down with explicit markers -- ✅ Cost-effective (uses fast models for extraction) - -## Implementation Details - -### Incremental Processing - -The system only processes **new content** added since the last commit: - -1. Uses `git diff HEAD ` to get changes -2. Extracts only lines starting with `+` (added lines) -3. Feeds incremental content to AI agents -4. Updates summary sections non-destructively - -### Participant Agent Scripts - -Before any output writers run, the runner checks the matched rule for a `participants` list. Each entry is a Python script path that will be executed with the staged file: - -```yaml -rules: - feature_discussion_update: - participants: - - path: "agents/moderator.py" - - path: "agents/visualizer.py" - background: true - outputs: - self_append: - path: "{path}" - output_type: "feature_discussion_writer" -``` - -Key points: - -- Agent scripts live under `agents/` and reuse the shared SDK in `src/cascadingdev/agent/`. -- `AgentContext` exposes helpers like `read_text()` and `append_block()` to work safely against the repo root. -- `ProviderClient` calls the configured AI chain (Claude → Codex → Gemini) and can return free-form or JSON responses. -- The included moderator agent appends a single guided comment per commit, guarded by `` to remain idempotent. -- The visualizer agent watches for `@AI_visual` mentions, generates PlantUML diagrams, and links them back to the discussion. -- Add `background: true` for tool-style agents (researcher, visualizer) so the runner launches them asynchronously; their follow-up work lands in the working tree after the commit and can be included in the next commit. These service agents provide information only and intentionally omit `VOTE:` lines to avoid influencing promotion thresholds. - -Adding new personas is as simple as dropping a script into `agents/` and registering it in `.ai-rules.yml`. - -### Marker Block System - -Summary files use HTML comment markers for non-destructive updates. In addition to the content sections, a special state marker is used to persist structured data across runs. - -#### Persistent Summary State - -To enable robust, incremental updates, the system stores the aggregated state of questions, action items, decisions, and mentions in a JSON blob within a comment at the top of the summary file. - -```markdown - -``` - -**How it works:** - -1. Before processing new discussion content, the `summary.py` script reads this state blob from the `.sum.md` file. -2. New items extracted from the latest discussion changes are merged with the existing state. -3. This merging logic deduplicates items and updates their status (e.g., a question moving from `OPEN` to `ANSWERED`). -4. The updated state is written back to the marker. -5. The visible summary sections (like `OPEN_QUESTIONS`) are then regenerated from this canonical state. - -This approach prevents information loss and ensures that the summary accurately reflects the cumulative history of the discussion, even as the discussion file itself grows. - -#### Content Sections - -```markdown - -## Section Header - - -``` - -**Sections:** -- `VOTES` - Vote counts and participants -- `DECISIONS` - ADR-style decisions -- `OPEN_QUESTIONS` - Unanswered questions -- `AWAITING` - Unresolved @mentions -- `ACTION_ITEMS` - TODO → ASSIGNED → DONE -- `TIMELINE` - Chronological updates (future) -- `LINKS` - Related PRs/commits (future) - -### Implementation Stage Enhancements - -- Checkbox items in `implementation.discussion.md` are parsed and normalised into - `implementation/tasks.md` and the summary's **Tasks** block. -- Promotion to testing now checks two conditions before updating YAML frontmatter: - 1. All detected checkboxes are `[x]`. - 2. At least one human participant has a `VOTE: READY`. -- These safeguards run even when AI providers fail, ensuring the stage cannot - advance on agent votes alone or with unfinished work. - -### Error Handling - -The workflow is **non-blocking**: -- Always exits 0 (never blocks commits) -- Prints warnings to stderr for missing dependencies -- Falls back to Phase 1 (votes only) if API key missing -- Continues on individual agent failures - -### Testing - -```bash -# Run all tests -pytest tests/ - -# Test vote parsing -pytest tests/test_workflow.py -v - -# Manual test in a project -cd /path/to/cascadingdev-project -echo "- Test: Comment with vote. VOTE: READY" >> Docs/features/test/discussions/test.discussion.md -git add Docs/features/test/discussions/test.discussion.md -git commit -m "Test workflow" # Triggers automation -``` - -## Configuration - -### AI Provider Options - -**1. Claude CLI (Default)** -```bash -# No configuration needed if you have 'claude' command available -# The system defaults to: claude -p '{prompt}' - -# To customize: -git config cascadingdev.aicommand "claude -p '{prompt}'" -``` - -**2. Gemini CLI** -```bash -git config cascadingdev.aiprovider "gemini-cli" -git config cascadingdev.aicommand "gemini '{prompt}'" -``` - -**3. OpenAI Codex CLI** -```bash -git config cascadingdev.aiprovider "codex-cli" -git config cascadingdev.aicommand "codex '{prompt}'" -``` - -**4. Direct API (Anthropic)** -```bash -pip install anthropic -export ANTHROPIC_API_KEY="sk-ant-..." -# CLI commands will fallback to API if available -``` - -**5. Custom AI Command** -```bash -# Use any command that reads a prompt and returns JSON -git config cascadingdev.aicommand "my-ai-tool --prompt '{prompt}' --format json" -``` - -### Disabling AI Features - -Simply don't configure any AI provider. The system will: -- Log a warning: `[agents] warning: No AI provider configured, skipping AI processing` -- Continue with Phase 1 (vote tracking only) -- Still extract @mentions (doesn't require AI) - -## Future Enhancements - -🚧 **Phase 3 (Planned):** -- Timeline auto-population from git commits -- Link tracking (related PRs, commits) -- Multi-file decision tracking -- Slack/Discord notification integration -- Summary diffs between commits -- Natural language summary generation - -## Troubleshooting - -### "agents module not available" - -**Cause:** Import path issue when workflow.py runs from pre-commit hook. - -**Solution:** Already fixed in workflow.py with dual import style: -```python -try: - from automation import agents -except ImportError: - import agents # Fallback for different execution contexts -``` - -### "No AI provider configured" - -**Cause:** No AI CLI command or API key configured. - -**Solution:** Choose one: -```bash -# Option 1: Use Claude CLI (default) -git config cascadingdev.aicommand "claude -p '{prompt}'" - -# Option 2: Use Gemini CLI -git config cascadingdev.aiprovider "gemini-cli" -git config cascadingdev.aicommand "gemini '{prompt}'" - -# Option 3: Use Anthropic API -pip install anthropic -export ANTHROPIC_API_KEY="sk-ant-..." -``` - -Or accept Phase 1 only (votes only, still useful) - -### "AI command failed with code X" - -**Cause:** AI CLI command returned error. - -**Solution:** -1. Test command manually: `claude -p "test prompt"` -2. Check command is in PATH: `which claude` -3. Verify command syntax in config: `git config cascadingdev.aicommand` -4. Check stderr output in warning message for details - -### Summary sections not updating - -**Cause:** Markers might be malformed or missing. - -**Solution:** -1. Check summary file has proper markers (see "Marker Block System") -2. Regenerate from template if needed -3. File should be created automatically by pre-commit hook - -### Votes not being detected - -**Cause:** Format doesn't match parser expectations. - -**Solution:** Ensure format is: -```markdown -- ParticipantName: Comment text. VOTE: READY -``` - -Common issues: -- Missing `- ` bullet -- Missing `: ` after name -- Typo in vote value (must be READY, CHANGES, or REJECT) -- Multiple participants on one line (not supported) - -## See Also - -- [DESIGN.md](DESIGN.md) - Overall system architecture -- [CLAUDE.md](../CLAUDE.md) - Guide for AI assistants -- [USER_GUIDE.md](../assets/templates/USER_GUIDE.md) - User-facing documentation diff --git a/docs/DESIGN.md b/docs/DESIGN.md deleted file mode 100644 index 199357f..0000000 --- a/docs/DESIGN.md +++ /dev/null @@ -1,3378 +0,0 @@ -# CascadingDev - AI–Human Collaboration System -## Process & Architecture Design Document (v2.1) -- Feature ID: FR_2025-10-21_initial-feature-request -- Status: Design Approved (Ready for Implementation) -- Date: 2025-10-21 (Updated: 2025-11-02) -- Owners: Rob (maintainer), AI_Moderator (process steward) -- Contributors: AI_Claude, AI_Deepseek, AI_Junie, AI_Chat-GPT, AI_GitHubCopilot - -## Implementation Status (2025-11-02) - -### ✅ Currently Implemented (Milestone M0-M1) -- **Stage 1: Request** - Feature request creation and automation -- **Stage 2: Feature Discussion** - Discussion file generation, voting, summaries -- **Cascading Rules System** - .ai-rules.yml loading and merging -- **AI Patch Generation** - Claude API integration, diff application -- **Multi-Provider Fallback** - Claude → Codex → Gemini chain with model hints -- **AI Normalization** - Natural conversation → structured data extraction -- **Vote Tracking** - VOTE: line parsing, summary updates -- **Pre-commit Hook** - Automation orchestration, file staging -- **Setup Script** - Project initialization with Ramble GUI - -### 🚧 In Progress (Current Focus) -- **Stage 3: Design Discussion Gate** - Currently skipped, being implemented - - Will create design.discussion.md when feature status = READY_FOR_DESIGN - - Will maintain design/design.md document - - Requires design_gate_writer and design_discussion_writer rules - -### 📋 Planned (Milestones M2-M4) -- **Stage 4: Implementation Discussion** - Task tracking, human approval gates -- **Stage 5: Testing Discussion** - Test planning, bug sub-cycles -- **Stage 6: Review Discussion** - Final review, release promotion -- **Stage 7: Release** - Changelog generation, version tagging -- **Moderator Protocol** - Nudge system, escalation paths -- **Bug Sub-Cycles** - Integrated bug tracking within features - ---- - -## Table of Contents -- Executive Summary -- Repository Layout -- Stage Model & Operational Procedure -- Voting, Quorum & Etiquette -- Cascading Rules System -- Orchestration Architecture -- Moderator Protocol -- Error Handling & Resilience -- Security & Secrets Management -- Performance & Scale Considerations -- Testing Strategy -- Implementation Plan -- Risks & Mitigations -- Template Evolution -- Roles & Agent Personas -- Glossary -- Appendices - -## Executive Summary - -We are implementing a **Git-native**, rules-driven workflow that enables seamless collaboration between humans and multiple AI agents across the entire software development lifecycle. The system uses cascading .ai-rules.yml configurations and a thin Bash pre-commit hook to automatically generate and maintain development artifacts (discussions, design docs, reviews, diagrams, plans). A Python orchestrator provides structured checks and status reporting while preserving the fast Bash execution path. - -**Scope clarification:** The document you are reading is the *CascadingDev system* design. It is **not** copied into user projects. End-users get a short `USER_GUIDE.md` and a `create_feature.py` tool; their **first feature request defines the project**, and its later design doc belongs to that project, not to CascadingDev. - -> *Git-Native Philosophy: Every conversation, decision, and generated artifact lives in the same version-controlled environment as the source code. There are no external databases, dashboards, or SaaS dependencies required for the core workflow. - -### Objective: -Establish a reproducible, self-documenting workflow where 90% of documentation, status artifacts and code changes are generated automatically from development discussions, while maintaining human oversight for all promotion gates and releases. - -### Core Principles -- Lightweight & Fast: Everything stored in Git as Markdown; minimal external dependencies -- Single Source of Truth: Repository contains all conversations, decisions, and code artifacts -- Self-Driving with Human Safety: AI agents can propose and vote; humans must approve critical stages -- Deterministic & Reversible: All automated actions are diffed, logged, and easily revertible -- Composable Rules: Nearest-folder precedence via cascading .ai-rules.yml configurations - -### Innovative Features -- Stage-Per-Discussion Model: Separate conversation threads for each development phase -- Automated Artifact Generation: Discussions automatically drive corresponding documentation -- Integrated Bug Sub-Cycles: Test failures automatically spawn bug reports with their own mini-lifecycle -- Intelligent Promotion Gates: Status-based transitions with configurable voting thresholds -- Multi-Agent Role Specialization: Different AI personas with stage-specific responsibilities - -### System Overview: -The architecture consists of a lightweight Bash pre-commit hook for artifact generation, a Python orchestrator for state evaluation and policy enforcement, and optional adapters for model and API integrations (Claude, Gitea, etc.). Together they form a layered, rule-driven automation stack. - -```text -Human → Git Commit → Pre-commit Hook → AI Generator → Markdown Artifact - ↑ ↓ - Orchestrator ← Discussion Summaries ← AI Moderator -``` - -## Repository Layouts - -This section clarifies three different directory structures that are easy to confuse: - -### Terminology -- **CascadingDev Repo** — The tooling project (this repository) that builds installers -- **Install Bundle** — The distributable artifact created by `tools/build_installer.py` -- **User Project** — A new repository scaffolded when a user runs the installer - ---- - -### A) CascadingDev Repository (Tooling Source) - -This is the development repository where CascadingDev itself is maintained. - -```text -CascadingDev/ # This repository -├─ automation/ # Workflow automation scripts -│ ├─ runner.py # AI rules orchestrator invoked from hooks -│ ├─ config.py # Cascading .ai-rules loader -│ ├─ patcher.py # Diff generation + git apply helpers -│ └─ workflow.py # Vote parsing, status reporting -├─ src/cascadingdev/ # Core Python modules -│ ├─ cli.py # Developer CLI (cdev command) -│ ├─ setup_project.py # Installer script (copied to bundle) -│ └─ utils.py # Version management, utilities -├─ assets/ # Single source of truth for shipped files -│ ├─ hooks/ -│ │ └─ pre-commit # Git hook template (bash script) -│ ├─ templates/ # Templates copied to user projects -│ │ ├─ USER_GUIDE.md # Daily usage guide -│ │ ├─ feature_request.md # Feature request template -│ │ ├─ feature.discussion.md # Discussion template -│ │ ├─ feature.discussion.sum.md # Summary template -│ │ ├─ design_doc.md # Design document template -│ │ ├─ root_gitignore # Root .gitignore template -│ │ ├─ process/ -│ │ │ └─ policies.yml # Machine-readable policies -│ │ └─ rules/ -│ │ ├─ root.ai-rules.yml # Root cascading rules -│ │ └─ features.ai-rules.yml # Feature-level rules -│ └─ runtime/ # Scripts copied to bundle & user projects -│ ├─ ramble.py # GUI for feature creation (PySide6/PyQt5) -│ ├─ create_feature.py # CLI for feature creation -│ └─ .gitignore.seed # Gitignore seed patterns -├─ tools/ # Build and test automation -│ ├─ build_installer.py # Creates install bundle -│ ├─ smoke_test.py # Basic validation tests -│ └─ bundle_smoke.py # End-to-end installer testing -├─ install/ # Build output directory (git-ignored) -│ └─ cascadingdev-/ # Generated installer bundle (see section B) -├─ docs/ # System documentation -│ ├─ DESIGN.md # This comprehensive design document -│ └─ INSTALL.md # Installation instructions -├─ tests/ # Test suite (planned, not yet implemented) -│ ├─ unit/ -│ ├─ integration/ -│ └─ bin/ -├─ VERSION # Semantic version (e.g., 0.1.0) -├─ pyproject.toml # Python package configuration -├─ README.md # Public-facing project overview -└─ CLAUDE.md # AI assistant guidance - -> **Maintainer vs. user tooling:** the `cdev` CLI (in `src/cascadingdev/`) is only used to build/test the CascadingDev installer. Once a user bootstraps a project, all automation is driven by the pre-commit hook invoking `automation/runner.py` under the control of the project's own `.ai-rules.yml` files. - -FUTURE (planned but not yet implemented): -├─ automation/ # 🚧 M1: Orchestration layer -│ ├─ workflow.py # Status reporting, vote parsing -│ ├─ adapters/ -│ │ ├─ claude_adapter.py # AI model integration -│ │ └─ gitea_adapter.py # Gitea API integration -│ └─ agents.yml # Agent role definitions -``` - -**Purpose:** Development, testing, and building the installer. The `assets/` directory is the single source of truth for all files shipped to users. - ---- - -### B) Install Bundle (Distribution Artifact) - -This is the self-contained, portable installer created by `tools/build_installer.py`. - -```text -cascadingdev-/ # Distributable bundle -├─ setup_cascadingdev.py # Installer entry point (stdlib only) -├─ ramble.py # GUI for first feature (optional) -├─ create_feature.py # CLI tool for creating features -├─ assets/ # Embedded resources -│ ├─ hooks/ -│ │ └─ pre-commit # Pre-commit hook template -│ └─ templates/ # All templates from source assets/ -│ ├─ USER_GUIDE.md -│ ├─ feature_request.md -│ ├─ feature.discussion.md -│ ├─ feature.discussion.sum.md -│ ├─ design_doc.md -│ ├─ root_gitignore -│ ├─ process/ -│ │ └─ policies.yml -│ └─ rules/ -│ ├─ root.ai-rules.yml -│ └─ features.ai-rules.yml -├─ INSTALL.md # Bundle-local instructions -└─ VERSION # Version metadata - -``` - -**Purpose:** End-user distribution. Can be zipped and shared. Requires only Python 3.10+ stdlib (PySide6 optional for GUI). - -**Rationale:** Minimal, auditable, portable. No external dependencies for core functionality. Users can inspect all files before running. - ---- - -### C) User Project (Generated by Installer) - -This is the structure created when a user runs `setup_cascadingdev.py --target /path/to/project`. - -```text -my-project/ # User's application repository -├─ .git/ # Git repository -│ └─ hooks/ -│ └─ pre-commit # Installed automatically from bundle -├─ .gitignore # Generated from root_gitignore template -├─ .ai-rules.yml # Root cascading rules (from templates/rules/) -├─ USER_GUIDE.md # Daily workflow reference -├─ ramble.py # Copied from bundle (optional GUI helper) -├─ create_feature.py # Copied from bundle (CLI tool) -├─ Docs/ # Documentation and feature tracking -│ ├─ features/ # All features live here -│ │ ├─ .ai-rules.yml # Feature-level cascading rules -│ │ └─ FR_YYYY-MM-DD_/ # Individual feature folders -│ │ ├─ request.md # Original feature request -│ │ └─ discussions/ # Stage-specific conversation threads -│ │ ├─ feature.discussion.md # Feature discussion -│ │ ├─ feature.discussion.sum.md # Auto-maintained summary -│ │ ├─ design.discussion.md # Design discussion -│ │ ├─ design.discussion.sum.md # Auto-maintained summary -│ │ ├─ implementation.discussion.md # Implementation tracking -│ │ ├─ implementation.discussion.sum.md -│ │ ├─ testing.discussion.md # Test planning -│ │ ├─ testing.discussion.sum.md -│ │ ├─ review.discussion.md # Final review -│ │ └─ review.discussion.sum.md -│ │ ├─ design/ # Design artifacts (created during design stage) -│ │ │ ├─ design.md # Evolving design document -│ │ │ └─ diagrams/ # Architecture diagrams -│ │ ├─ implementation/ # Implementation artifacts -│ │ │ ├─ plan.md # Implementation plan -│ │ │ └─ tasks.md # Task checklist -│ │ ├─ testing/ # Testing artifacts -│ │ │ ├─ testplan.md # Test strategy -│ │ │ └─ checklist.md # Test checklist -│ │ ├─ review/ # Review artifacts -│ │ │ └─ findings.md # Code review findings -│ │ └─ bugs/ # Bug sub-cycles (future) -│ │ └─ BUG_YYYYMMDD_/ -│ │ ├─ report.md -│ │ ├─ discussion.md -│ │ └─ fix/ -│ │ ├─ plan.md -│ │ └─ tasks.md -│ ├─ discussions/ # Global discussions (future) -│ │ └─ reviews/ # Code reviews from hook -│ └─ diagrams/ # Auto-generated diagrams (future) -│ └─ file_diagrams/ # PlantUML from source files -├─ process/ # Process configuration -│ ├─ policies.yml # Machine-readable policies (voting, gates) -│ └─ templates/ # Local template overrides (optional) -├─ src/ # User's application source code -│ └─ (user's code) -└─ tests/ # User's test suite - ├─ unit/ - └─ integration/ - -FUTURE (not currently created, planned for M1+): -├─ automation/ # 🚧 M1: Orchestration layer -│ ├─ workflow.py # Vote parsing, status reporting -│ ├─ adapters/ # Model and platform integrations -│ └─ agents.yml # Agent role configuration -``` - -**Purpose:** This is the user's actual project repository where they develop their application while using the CascadingDev workflow. - -**Key Points:** -- The first feature request defines the entire project's purpose -- All discussions are version-controlled alongside code -- Pre-commit hook maintains summary files automatically -- Templates can be overridden locally in `process/templates/` -- The `automation/` directory is planned but not yet implemented (M1) - ---- - -## Installation & Distribution Architecture - -### First-Run Flow (User's Project Initialization) - -User runs: -```bash -python setup_cascadingdev.py --target /path/to/users-project [--no-ramble] [--provider mock] -``` - -**Installer actions:** -- Creates standard folders (Docs/, process/templates/, etc.) -- Copies templates, ramble.py, and create_feature.py into the user project -- Initializes git (main branch), writes `.gitignore` -- Installs pre-commit hook -- Optionally launches Ramble (unless --no-ramble) to help collect first Feature Request -- Writes a concise USER_GUIDE.md into the user project root for day-to-day use - -**Seeds:** -``` -Docs/features/FR__initial-feature-request/request.md -Docs/features/.../discussions/feature.discussion.md -Docs/features/.../discussions/feature.discussion.sum.md -``` - -Initial commit message: “bootstrap Cascading Development scaffolding”. - -**Fallback:** If Ramble JSON isn’t returned, installer prints to stderr and optionally falls back to terminal prompts. - -Important: The CascadingDev DESIGN.md is not copied into user projects. The first feature’s design doc (created later at the design stage) becomes the project’s own design document. - -### Pre-Commit Hook (v1 behavior) -- Fast regex secret scan on staged diffs -- Ensures each `*.discussion.md` has a companion `*.discussion.sum.md` -- Non-blocking status call to `automation/workflow.py --status` - -Policy: v1 is non-blocking; blocking checks are introduced gradually in later versions. - ---- - -## Template META System & Ramble Integration - -### Overview - -CascadingDev includes a sophisticated **template metadata system** that allows templates to be self-describing. This enables dynamic GUI generation, field validation, and flexible template rendering without hardcoding form structures in the installer. - -**Status**: ✅ **Fully implemented** (v0.1.0) - -### Template META Format - -Templates can include JSON metadata inside HTML comments at the top of the file: - -```markdown - - -# Feature Request: {Title} - -**Intent**: {Intent} -**Summary**: {Summary} - -**Meta**: FeatureId: {FeatureId} • Created: {CreatedDate} -``` - -### META Fields Reference - -| Field | Type | Purpose | Example | -|-------|------|---------|---------| -| `kind` | string | Template type identifier | `"feature_request"` | -| `ramble_fields` | array | Field definitions for Ramble GUI | See below | -| `criteria` | object | Validation rules per field | `{"Title": "camelCase, <= 24 chars"}` | -| `hints` | array | User guidance prompts | `["What is it called?"]` | -| `tokens` | array | List of available placeholder tokens | `["FeatureId", "Title"]` | - -### ramble_fields Specification - -Each field in `ramble_fields` is an object with: - -```json -{ - "name": "FieldName", // Required: field identifier - "hint": "display hint", // Optional: shown to user as guidance - "default": "defaultValue" // Optional: pre-filled value -} -``` - -**Example:** -```json -"ramble_fields": [ - {"name": "Title", "hint": "camelCase, ≤24 chars", "default": "initialProjectDesign"}, - {"name": "Intent"}, - {"name": "ProblemItSolves"}, - {"name": "BriefOverview"}, - {"name": "Summary", "hint": "≤2 sentences"} -] -``` - -### How META is Processed - -**In `setup_project.py`** (`src/cascadingdev/setup_project.py:64-115`): - -1. **Parsing** (`load_template_with_meta()`): - ```python - meta, body = load_template_with_meta(template_path) - # meta = {"ramble_fields": [...], "criteria": {...}, ...} - # body = template text without META comment - ``` - -2. **Extraction** (`meta_ramble_config()`): - ```python - fields, defaults, criteria, hints = meta_ramble_config(meta) - # fields = ["Title", "Intent", "Summary", ...] - # defaults = {"Title": "initialProjectDesign"} - # criteria = {"Title": "camelCase, <= 24 chars"} - # hints = ["What is it called?", ...] - ``` - -3. **Rendering** (`render_placeholders()`): - ```python - values = {"Title": "myFeature", "FeatureId": "FR_2025-10-30_...", ...} - rendered = render_placeholders(body, values) - # Replaces {Token} and {{Token}} with actual values - ``` - -### Token Replacement Rules - -The `render_placeholders()` function supports two-pass replacement: - -1. **First pass**: Replace `{{Token}}` (double braces) - for tokens that shouldn't be re-processed -2. **Second pass**: Replace `{Token}` (single braces) using Python's `.format_map()` - -**System-provided tokens:** -- `{FeatureId}` - Generated feature ID (e.g., `FR_2025-10-30_initial-feature-request`) -- `{CreatedDate}` - Current date in `YYYY-MM-DD` format -- `{Title}`, `{Intent}`, etc. - User-provided field values - ---- - -## Ramble: AI-Powered Feature Capture GUI - -### Overview - -**Ramble** is a sophisticated PySide6/PyQt5 GUI application that helps users articulate feature requests through AI-assisted structured input. It supports multiple AI providers, generates PlantUML diagrams, and returns validated JSON output. - -**Status**: ✅ **Fully implemented** (v0.1.0) - -**Location**: `assets/runtime/ramble.py` (copied to user projects and install bundle) - -### Key Features - -1. **Multi-Provider Architecture** - Pluggable AI backends -2. **Dynamic Field Generation** - Driven by template META -3. **Field Locking** - Lock fields to preserve context across regenerations -4. **PlantUML Integration** - Auto-generate and render architecture diagrams -5. **Validation Criteria** - Per-field rules from template metadata -6. **Graceful Fallback** - Terminal prompts if GUI fails - -### Supported Providers - -| Provider | Status | Description | Usage | -|----------|--------|-------------|-------| -| **mock** | ✅ Stable | No external calls, derives fields from ramble text | Default, no setup required | -| **claude** | ✅ Stable | Claude CLI integration via subprocess | Requires `claude` CLI in PATH | - -**Provider Selection:** -```bash -# Mock provider (no AI, instant) -python ramble.py --provider mock --fields Title Summary - -# Claude CLI provider -python ramble.py --provider claude \ - --claude-cmd /path/to/claude \ - --fields Title Summary Intent -``` - -### Provider Protocol - -All providers implement the `RambleProvider` protocol: - -```python -class RambleProvider(Protocol): - def generate( - self, - *, - prompt: str, # User's base prompt - ramble_text: str, # User's freeform notes - fields: List[str], # Required field names - field_criteria: Dict[str, str], # Validation rules per field - locked_context: Dict[str, str], # Previously locked field values - ) -> Dict[str, Any]: - """ - Returns: - { - "summary": str, - "fields": Dict[str, str], - "uml_blocks": List[Tuple[str, Optional[bytes]]], - "image_descriptions": List[str] - } - """ - ... -``` - -### Mock Provider - -**Purpose**: Fast, deterministic testing and offline use. - -**Behavior**: -- Derives summary from last 25 words of ramble text -- Creates placeholder fields with word count -- Generates simple actor-system UML diagram -- Returns generic image descriptions - -**Example Output**: -```python -{ - "summary": "User wants to track metrics and export them.", - "fields": { - "Title": "Title: Derived from ramble (42 words). [criteria: camelCase, <=24 chars]", - "Intent": "Intent: Derived from ramble (42 words).", - }, - "uml_blocks": [("@startuml\nactor User\n...\n@enduml", None)], - "image_descriptions": ["Illustrate the core actor..."] -} -``` - -### Claude CLI Provider - -**Purpose**: Production-quality AI-generated structured output. - -**Setup Requirements**: -```bash -# Install Claude CLI (npm) -npm install -g @anthropics/claude-cli - -# Or provide custom path -python ramble.py --provider claude --claude-cmd /custom/path/to/claude -``` - -**Features**: -- Spawns `claude` subprocess with structured prompt -- Includes locked field context in prompt -- Enforces per-field criteria -- Extracts PlantUML blocks from response -- Timeout protection (default 120s) -- Debug logging to `/tmp/ramble_claude.log` - -**Constructor Options**: -```python -ClaudeCLIProvider( - cmd="claude", # Command name or path - extra_args=[], # Additional CLI args - timeout_s=120, # Subprocess timeout - tail_chars=8000, # Max response length - use_arg_p=True, # Use -p flag for prompt - debug=False, # Enable debug logging - log_path="/tmp/ramble_claude.log" -) -``` - -**Prompt Structure**: -The provider builds a comprehensive prompt including: -1. User's base prompt -2. Locked field context (from previously locked fields) -3. User's ramble notes -4. Required field list with criteria -5. PlantUML and image description requests -6. JSON output format specification - -### Integration with Installer - -**In `setup_project.py:151-218`** (`run_ramble_and_collect()`): - -```python -def run_ramble_and_collect(target: Path, provider: str = "mock", claude_cmd: str = "claude"): - # 1. Load template META to get field configuration - fr_tmpl = INSTALL_ROOT / "assets" / "templates" / "feature_request.md" - meta, _ = load_template_with_meta(fr_tmpl) - field_names, _defaults, criteria, hints = meta_ramble_config(meta) - - # 2. Build dynamic Ramble command from META - args = [ - sys.executable, str(ramble), - "--provider", provider, - "--fields", *field_names, # From template META - ] - - if criteria: - args += ["--criteria", json.dumps(criteria)] - if hints: - args += ["--hints", json.dumps(hints)] - - # 3. Launch Ramble, capture JSON output - proc = subprocess.run(args, capture_output=True, text=True) - - # 4. Parse JSON or fall back to terminal prompts - try: - return json.loads(proc.stdout) - except: - # Terminal fallback: collect fields via input() - return collect_via_terminal() -``` - -### Ramble GUI Workflow - -1. **User writes freeform notes** in the "Ramble" text area -2. **Clicks "Generate"** → Provider processes ramble text -3. **Review generated fields** → Edit as needed -4. **Lock important fields** → Prevents overwrite on regenerate -5. **Regenerate if needed** → Locked fields feed back as context -6. **Review PlantUML diagrams** → Auto-rendered if plantuml CLI available -7. **Click "Submit"** → Returns JSON to installer - -**Output Format**: -```json -{ - "summary": "One or two sentence summary", - "fields": { - "Title": "metricsExportFeature", - "Intent": "Enable users to track and export usage metrics", - "ProblemItSolves": "Currently no way to analyze usage patterns", - "BriefOverview": "Add metrics collection and CSV/JSON export", - "Summary": "Track usage metrics and export to various formats." - } -} -``` - -### Terminal Fallback - -If Ramble GUI fails (missing PySide6, JSON parse error, etc.), the installer falls back to terminal input: - -```python -def ask(label, default=""): - try: - v = input(f"{label}: ").strip() - return v or default - except EOFError: - return default - -fields = { - "Title": ask("Title (camelCase, <=24 chars)", "initialProjectDesign"), - "Intent": ask("Intent", "—"), - "Summary": ask("One- or two-sentence summary", ""), -} -``` - -### Adding New Providers - -To add a new provider (e.g., `deepseek`, `openai`): - -1. **Create provider class** in `ramble.py`: - ```python - class DeepseekProvider: - def generate(self, *, prompt, ramble_text, fields, field_criteria, locked_context): - # Call Deepseek API - response = call_deepseek_api(...) - return { - "summary": ..., - "fields": {...}, - "uml_blocks": [...], - "image_descriptions": [...] - } - ``` - -2. **Register in CLI parser**: - ```python - p.add_argument("--provider", - choices=["mock", "claude", "deepseek"], # Add here - default="mock") - ``` - -3. **Instantiate in main()**: - ```python - if args.provider == "deepseek": - provider = DeepseekProvider(api_key=os.getenv("DEEPSEEK_API_KEY")) - ``` - -### Advanced Features - -**PlantUML Support**: -- Ramble extracts `@startuml...@enduml` blocks from provider responses -- Auto-renders to PNG if `plantuml` CLI available -- Falls back to text display if rendering fails - -**Image Generation** (Optional): -- Supports Stability AI and Pexels APIs -- Requires API keys via environment variables -- Displays images in GUI if generated - -**Field Locking**: -- Checkbox next to each field -- Locked fields are highlighted and included in next generation prompt -- Enables iterative refinement without losing progress - -**Criteria Validation**: -- Displayed alongside each field as hints -- Passed to AI provider to enforce constraints -- No automatic validation (relies on AI compliance) - -### Configuration Examples - -**Basic usage (mock provider)**: -```bash -python ramble.py \ - --fields Title Intent Summary \ - --prompt "Describe your feature idea" -``` - -**Production usage (Claude)**: -```bash -python ramble.py \ - --provider claude \ - --claude-cmd ~/.npm-global/bin/claude \ - --fields Title Intent ProblemItSolves BriefOverview Summary \ - --criteria '{"Title":"camelCase, <=24 chars","Summary":"<=2 sentences"}' \ - --hints '["What is it?","Who benefits?","What problem?"]' \ - --prompt "Describe your initial feature request" -``` - -**Installer integration**: -```bash -python setup_cascadingdev.py \ - --target /path/to/project \ - --provider claude \ - --claude-cmd /usr/local/bin/claude -``` - -### Benefits of META + Ramble System - -1. **No Hardcoding**: Field lists and validation rules live in templates -2. **Dynamic Forms**: GUI adapts to template changes automatically -3. **Consistent UX**: Same Ramble workflow for all template types -4. **Extensible**: Add new providers without changing core logic -5. **Offline Capable**: Mock provider works without network -6. **AI-Assisted**: Users get help articulating complex requirements -7. **Reversible**: All input is stored in git, easily editable later - -### Limitations & Future Work - -**Current Limitations**: -- No automatic field validation (relies on AI compliance) -- PlantUML rendering requires external CLI tool -- Claude provider requires separate CLI installation -- No streaming/incremental updates during generation - -**Potential Enhancements** (not yet planned): -- Native API providers (no CLI subprocess) -- Real-time field validation -- Multi-turn conversation support -- Provider comparison mode (generate with multiple providers) -- Template validator that checks META integrity - ---- - -### Build & Release Process (repeatable) - -Goal: deterministic “unzip + run” artifact for each version. - -**Always rebuild after edits** -```bash -# Rebuild bundle every time you change assets/ or installer logic -python tools/build_installer.py - -# Run ONLY the bundled copy -python install/cascadingdev-*/setup_cascadingdev.py --target /path/to/new-project -``` - -**6.1 Versioning** -- Update `VERSION` (semver): `MAJOR.MINOR.PATCH` -- Tag releases in git to match `VERSION` - -**6.2 Build the installer bundle** -```bash -python3 tools/build_installer.py -# Output: install/cascadingdev-/ -``` - -**6.3 Smoke test the bundle** (verifies minimal Python+git system compatibility) -```bash -python3 install/cascadingdev-/setup_cascadingdev.py --target /tmp/my-users-project --no-ramble -# then again without --no-ramble once GUI deps are ready -``` - -**6.4 Package for distribution** -```bash -cd install -zip -r cascadingdev-.zip cascadingdev- -``` - -**Success criteria** -- Bundle runs on a clean machine with only Python + git -- Creates expected files, installs hook, commits once -- Optional GUI capture works under venv with PySide6/PyQt5 - - -### Environment Guidance (Ramble GUI) - -Recommended: create and activate a local virtualenv before running the installer. - -```bash -python3 -m venv .venv -source .venv/bin/activate -python -m pip install --upgrade pip wheel -python -m pip install PySide6 # or PyQt5 -``` - -The installer calls `sys.executable` for `ramble.py`, so using the venv’s Python ensures GUI dependencies are found. - -If GUI unavailable, run with `--no-ramble` or rely on terminal fallback. - - -### Idempotency & Flags - -Installer flags: -- `--target` (required): destination path for the user’s project -- `--no-ramble`: skip GUI prompt; seed using templates (or terminal fallback) -- `--provider`: specify Ramble provider (default: mock) - -Future flags (planned): -- `--force`: overwrite existing files (or back up) -- `--copy-only`: skip git init and hook install -- `--templates /path`: override shipped templates - - -### Reproducibility Playbook - -To recreate the same result later or on another machine: -```bash -git checkout -python3 tools/build_installer.py -python3 install/cascadingdev-/setup_cascadingdev.py --target -``` - -(Optional) activate venv and install PySide6 before running if you want the GUI. - - -### Roadmap (post-v0.1) -- Expand pre-commit checks (summary normalization, rule validation) -- CLI status report for current staged discussions and votes -- Provide a developer CLI (`cdev init …`) that reuses installer logic -- Unit tests for scaffolding helpers (`src/cascadingdev/*`) -- Template override mechanism (`--templates`) - - -### INSTALL.md Template for Bundles - -The builder should emit an `INSTALL.md` alongside the installer bundle: - -```markdown -# CascadingDev Installer - -## Requirements -- Python 3.10+ and git -- (Optional) PySide6 or PyQt5 for Ramble GUI - -## Quick start -```bash -python setup_cascadingdev.py --target /path/to/users-project -``` - -### Options -- \`--no-ramble\`: skip GUI capture, use templates or terminal prompts -- \`--provider\`: Ramble provider (default: mock) - -### Steps performed -1. Creates standard folder layout in target -2. Copies templates, DESIGN.md, ramble.py -3. Initializes git and installs pre-commit hook -4. Launches Ramble (unless --no-ramble) -5. Seeds first feature request & discussions -6. Makes initial commit - -If GUI fails, use a virtualenv and \`pip install PySide6\`, or run with \`--no-ramble\`. - -This ensures every distributed bundle includes explicit usage instructions. - - -Note: Each stage discussion has a companion summary maintained automatically next to it to provide a live, scannable state of the thread. - -### Naming Conventions -- Feature Folder: Docs/features/FR_YYYY-MM-DD_/ -- Discussion Files: {stage}.discussion.md in discussions/ subfolder -- Discussion summary: {stage}.discussion.sum.md in discussions/ subfolder -- Bug Reports: bugs/BUG_YYYYMMDD_/ with standardized contents -- Source Files: Maintain existing patterns in src/ - -### Template Variables -Supported in path resolution: -- {basename} — filename with extension (e.g., design.discussion.md) -- {name} — filename without extension (e.g., design.discussion) -- {ext} — file extension without dot (e.g., md) -- {date} — current date in YYYY-MM-DD -- {rel} — repository-relative path to the source file -- {dir} — directory containing the source file -- {feature_id} — nearest FR_* folder name (e.g., FR_2025-10-21_initial-feature-request) -- {stage} — stage inferred from discussion filename (.discussion.md), e.g., feature|design|implementation|testing|review - ---- - -## Automation AI Configuration - -### Overview - -The automation runner (`automation/runner.py` and `automation/patcher.py`) supports multiple AI providers with automatic fallback chains and intelligent model selection based on task complexity. This system balances cost, speed, and quality by routing simple tasks to fast models and complex tasks to premium models. - -### Configuration Architecture - -**Central Configuration:** `config/ai.yml` - -This file is copied to all generated projects and provides a single source of truth for AI provider preferences. It defines three command chains optimized for different use cases: - -```yaml -version: 1 - -runner: - # Default command chain (balanced speed/quality) - command_chain: - - "claude -p" - - "codex --model gpt-5" - - "gemini --model gemini-2.5-flash" - - # Fast command chain (optimized for speed/cost) - # Used when model_hint: fast in .ai-rules.yml - command_chain_fast: - - "claude -p" - - "codex --model gpt-5-mini" - - "gemini --model gemini-2.5-flash" - - # Quality command chain (optimized for complex tasks) - # Used when model_hint: quality in .ai-rules.yml - command_chain_quality: - - "claude -p" - - "codex --model o3" - - "gemini --model gemini-2.5-pro" - - sentinel: "CASCADINGDEV_NO_CHANGES" - -ramble: - default_provider: mock - providers: - mock: { kind: mock } - claude: { kind: claude_cli, command: "claude", args: [] } -``` - -### Model Hint System - -The `.ai-rules.yml` files can specify a `model_hint` field per rule to guide model selection: - -```yaml -rules: - feature_discussion_writer: - model_hint: fast # Simple vote counting - use fast models - instruction: | - Maintain feature discussion with votes... - - design_discussion_writer: - model_hint: quality # Complex architecture work - use quality models - instruction: | - Propose detailed architecture and design decisions... - - implementation_gate_writer: - model_hint: fast # Gate checking is simple - use fast models - instruction: | - Create implementation discussion when design is ready... -``` - -### Execution Flow - -1. **Participant Scripts** (`automation/runner.py:72-118`) - - Runner checks the matched rule for a `participants` list. - - Each script (e.g., `agents/moderator.py`) is executed with `--repo-root` and `--path`. - - Scripts leverage `src/cascadingdev/agent/` helpers to read the staged file and append one idempotent block before any generators run. - - Entries marked with `background: true` launch asynchronously so long-running tools (researcher, visualizer) can finish after the commit completes while keeping their informational updates outside the current commit. - -2. **Rule Evaluation** (`automation/runner.py:120-190`) - - Runner reads `.ai-rules.yml` and finds matching rule for staged file. - - Extracts `model_hint` from rule config (or output_type override). - - Passes hint to `generate_output(source_rel, output_rel, instruction, model_hint)`. - -3. **Prompt Construction** (`automation/patcher.py:371-406`) - - Patcher receives model_hint and builds prompt. - - Injects `TASK COMPLEXITY: FAST` or `TASK COMPLEXITY: QUALITY` line. - - This hint helps Claude subagents auto-select appropriate model. - -4. **Chain Selection** (`automation/patcher.py:63-77`) - - `ModelConfig.get_commands_for_hint(hint)` selects appropriate chain. - - Returns `command_chain_fast`, `command_chain_quality`, or default. - - Falls back to default chain if hint-specific chain is empty. - -5. **Fallback Execution** (`automation/patcher.py:409-450`) - - `call_model()` iterates through selected command chain left→right. - - First successful provider's output is used. - - Errors are logged; subsequent providers are tried. - - If all fail, error is raised (but pre-commit remains non-blocking). - -### Example Prompt with Hint - -When `model_hint: fast` is set, the generated prompt includes: - -``` -You are a specialized patch generator for CascadingDev automation. - -SOURCE FILE: Docs/features/FR_2025-11-01_auth/discussions/feature.discussion.md -OUTPUT FILE: Docs/features/FR_2025-11-01_auth/discussions/feature.discussion.sum.md -TASK COMPLEXITY: FAST - -=== SOURCE FILE CHANGES (staged) === -[diff content] - -=== CURRENT OUTPUT FILE CONTENT === -[summary file content] - -INSTRUCTIONS: -Update the vote summary section... -``` - -The `TASK COMPLEXITY: FAST` line signals to Claude's subagent system to select `cdev-patch` (Haiku) instead of `cdev-patch-quality` (Sonnet). - -### Supported Providers - -| Provider | CLI Tool | Fast Model | Default Model | Quality Model | -|----------|----------|------------|---------------|---------------| -| **Claude** | `claude -p` | Haiku (via subagent) | Auto-select | Sonnet (via subagent) | -| **OpenAI Codex** | `codex` | gpt-5-mini | gpt-5 | o3 | -| **Google Gemini** | `gemini` | gemini-2.5-flash | gemini-2.5-flash | gemini-2.5-pro | - -**Claude Subagent Setup:** -```bash -# Create ~/.claude/agents/cdev-patch.md and cdev-patch-quality.md -./tools/setup_claude_agents.sh - -# Verify installation -claude agents list -``` - -The setup script creates two subagent files with descriptions that include "MUST BE USED when TASK COMPLEXITY is FAST/QUALITY", enabling Claude's agent selection to respond to the hint in the prompt. - -### Implementation Modules - -**Core modules:** -- **`automation/ai_config.py`** - Configuration loader (`AISettings`, `RunnerSettings`, `RambleSettings`) - - `load_ai_settings(repo_root)` - Loads and validates config/ai.yml - - `RunnerSettings.get_chain_for_hint(hint)` - Returns appropriate command chain - - `parse_command_chain(raw)` - Splits "||" delimited command strings - -- **`automation/runner.py`** - Orchestrates rule evaluation and output generation - - Reads `model_hint` from cascaded rule config (line 94) - - Passes hint through to `generate_output()` (line 112) - - Supports output_type hint overrides (line 100-102) - -- **`automation/patcher.py`** - Generates and applies AI patches - - `ModelConfig.get_commands_for_hint(hint)` - Selects command chain (line 63-77) - - `build_prompt(..., model_hint)` - Injects TASK COMPLEXITY line (line 371-406) - - `call_model(..., model_hint)` - Executes chain with fallback (line 409-450) - -### Cost Optimization Examples - -**Before (all tasks use same model):** -- Vote counting: Sonnet @ $3/M tokens → $0.003/commit -- Design discussion: Sonnet @ $3/M tokens → $0.15/commit -- Gate checking: Sonnet @ $3/M tokens → $0.002/commit - -**After (intelligent routing):** -- Vote counting: Haiku @ $0.25/M tokens → $0.0002/commit (93% savings) -- Design discussion: Sonnet @ $3/M tokens → $0.15/commit (unchanged) -- Gate checking: Haiku @ $0.25/M tokens → $0.00017/commit (91% savings) - -For a project with 100 commits, savings of ~$0.30/commit × 70% simple tasks = **$21 saved** while maintaining quality for complex tasks. - -### Environment Overrides - -Users can temporarily override `config/ai.yml` via environment variables: - -```bash -# Override command for single commit -CDEV_AI_COMMAND="claude -p" git commit -m "message" - -# Chain multiple providers -CDEV_AI_COMMAND="claude -p || codex --model gpt-5" git commit -m "message" - -# Use only quality models for critical work -CDEV_AI_COMMAND="claude -p" git commit -m "critical refactor" -``` - -Environment variables take precedence over `config/ai.yml` but don't persist. - ---- - -## Stage Model & Operational Procedure - -### Complete Stage Lifecycle -```text -Request → Feature Discussion → Design Discussion → Implementation Discussion → Testing Discussion → Review Discussion → Release -``` -### Stage Overview - -| Stage | Primary File | Promotion Trigger | Human Gate | Key Artifacts Created | -|:------|:-------------|:------------------|:-----------|:----------------------| -| 1 Request | request.md | Created from template | – | feature.discussion.md, feature.discussion.sum.md | -| 2 Feature Discussion | discussions/feature.discussion.md | Votes → READY_FOR_DESIGN | – | design.discussion.md, design/design.md | -| 3 Design Discussion | discussions/design.discussion.md | Votes → READY_FOR_IMPLEMENTATION | – | implementation.discussion.md, implementation/plan.md, implementation/tasks.md | -| 4 Implementation Discussion | discussions/implementation.discussion.md | All required tasks complete → READY_FOR_TESTING | ✅ ≥1 human READY | testing.discussion.md, testing/testplan.md, testing/checklist.md | -| 5 Testing Discussion | discussions/testing.discussion.md | All tests pass → READY_FOR_REVIEW | – | review.discussion.md, review/findings.md | -| 6 Review Discussion | discussions/review.discussion.md | Human READY → READY_FOR_RELEASE | ✅ ≥1 human READY | Release notes, follow-up FRs/Bugs | -| 7 Release | – | Tag & changelog generated | ✅ maintainer | Changelog, version bump, rollback notes | - -### Stage 1: Request -#### Entry Criteria -- Docs/features/FR_*/request.md created from template -- Template completeness: intent, motivation, constraints, open questions - -#### Artifacts Generated -- request.md: Source feature request document - -#### Automated Actions -- Creates discussions/feature.discussion.md with standard header -- Adds Summary and Participation sections -- Appends initial AI comment with vote - -#### Exit Criteria -- Discussion file created and populated -- Ready for feature discussion phase - -### Stage 2: Feature Discussion -- File: discussions/feature.discussion.md - -#### Header Template - -```yaml ---- -type: discussion -stage: feature -status: OPEN # OPEN | READY_FOR_DESIGN | REJECTED -feature_id: FR_YYYY-MM-DD_ -stage_id: FR_YYYY-MM-DD__feature -created: YYYY-MM-DD - -promotion_rule: - allow_agent_votes: true - ready_min_eligible_votes: all - reject_min_eligible_votes: all - -participation: - instructions: | - - Append your input using the format: - - Name: Your Name - - Your comment... - - VOTE: READY|CHANGES|REJECT - - Agents/bots must prefix names with "AI_" - -voting: - values: [READY, CHANGES, REJECT] ---- -``` -#### Operational Flow -- Participants append comments ending with vote lines -- Latest vote per participant counts toward thresholds -- AI_Moderator tracks unanswered questions and missing votes -- When READY threshold met: status → READY_FOR_DESIGN - - _Promotion decisions obey thresholds in `process/policies.yml` (`voting.quorum`). The orchestrator re-evaluates eligibility before any status change._ - -#### Automated Actions on commit -- Appends AI comment with vote - - Moderates discussion - - Establishes objectives with consensus - - Delegates in-conversation tasks -- Creates/maintains discussions/feature.discussion.sum.md -> **Automation boundary:** All actions occur within the same Git commit and are never auto-committed by the orchestrator. - -#### Promotion Actions -- Creates discussions/design.discussion.md (OPEN) -- Creates design/design.md seeded from request + feature discussion - -### Stage 3: Design Discussion -- File: discussions/design.discussion.md - -#### Header - -```yaml ---- -type: discussion -stage: design -status: OPEN # OPEN | READY_FOR_IMPLEMENTATION | NEEDS_MORE_INFO -feature_id: FR_YYYY-MM-DD_ -stage_id: FR_YYYY-MM-DD__design -# ... same promotion_rule, participation, voting as feature ---- -``` - -#### Operational Flow -- AI_Architect updates design/design.md on each commit -- Design doc evolves with discussion: options, decisions, risks, acceptance criteria -- Participants vote on design completeness -- When READY threshold met: status → READY_FOR_IMPLEMENTATION - -_Promotion decisions obey thresholds in `process/policies.yml` (`voting.quorum`). The orchestrator re-evaluates eligibility before any status change._ - -#### Automated Actions on commit -- Appends AI comment with vote - - Moderates discussion - - establishes objective - - delegates in-conversation tasks -- Creates/maintains discussions/design.discussion.sum.md -- Creates/maintains design/design.md -- Creates/maintains design/diagrams/*.puml files if any are produced during the discussion. - -#### Design Document Structure -- Context & Goals -- Non-Goals & Constraints -- Options Considered with Trade-offs -- Decision & Rationale -- Architecture Diagrams -- Risks & Mitigations -- Measurable Acceptance Criteria - -#### Promotion Actions -- Creates discussions/implementation.discussion.md (OPEN) -- Creates implementation/plan.md and implementation/tasks.md -- Creates implementation/tasks.md - Tasks are checkboxes aligned to acceptance criteria - -### Stage 4: Implementation Discussion -- File: discussions/implementation.discussion.md - -#### Header - -```yaml ---- -type: implementation-discussion -stage: implementation -status: OPEN # OPEN | READY_FOR_TESTING -feature_id: FR_YYYY-MM-DD_ -promotion_rule: - allow_agent_votes: true - ready_min_eligible_votes: 1 - reject_min_eligible_votes: all ---- -``` - -#### Operational Flow -- AI_Implementer keeps the discussion and `implementation/tasks.md` in sync. -- Checkbox items (`- [ ]` / `- [x]`) are parsed directly from the discussion body and mirrored into tasks.md and the implementation summary. -- Promotion requires **all tasks checked** _and_ at least **one human** READY vote. - -#### Automated Actions on commit -- Appends AI_Implementer status comment summarising task progress and posting a READY/CHANGES vote automatically. -- Updates `discussions/implementation.discussion.sum.md` with votes, participants, and tasks. -- Mirrors checkbox state into `implementation/tasks.md` for downstream tooling. - -#### Task Management -- `implementation/tasks.md` is regenerated from the discussion; edit the discussion to change task state. -- Future work: capture PR/commit references and surface them in summaries. - -#### Promotion Actions -- When promoted to `READY_FOR_TESTING`, Stage 5 assets (testing discussion + checklists) are generated. - -### Stage 5: Testing Discussion -- File: discussions/testing.discussion.md - -#### Header - -```yaml -type: discussion -stage: testing -status: OPEN # OPEN | READY_FOR_REVIEW -feature_id: FR_YYYY-MM-DD_ -stage_id: FR_YYYY-MM-DD__testing -promotion_rule: - allow_agent_votes: true - ready_min_eligible_votes: all - reject_min_eligible_votes: all -``` ---- - - -#### Operational Flow -- AI_Tester syncs testing/checklist.md with discussion posts -- Parse result blocks: [RESULT] PASS/FAIL: description -- Mark corresponding checklist items pass/fail -- On test failure: auto-create bug report with full sub-cycle - -_Promotion decisions obey thresholds in `process/policies.yml` (`voting.quorum`). The orchestrator re-evaluates eligibility before any status change._ - -#### Automated Actions on commit -- Appends AI comment with vote - - Moderates discussion - - establishes testing objectives from testing/testplan.md and testing/checklist.md - - delegates testing tasks from testing/checklist.md -- Creates/maintains discussions/testing.discussion.sum.md -- Creates/maintains test/* files if any are produced during the discussion. - - -#### Bug Sub-Cycle Creation - -```text -bugs/BUG_YYYYMMDD_/ -├─ report.md # Steps, expected/actual, environment -├─ discussion.md # Bug discussion (OPEN) -├─ discussion.sum.md # Summary of Bug discussion -└─ fix/ - ├─ plan.md # Fix implementation plan - └─ tasks.md # Fix tasks checklist - └─ src/ -``` - -#### Bug Resolution Flow -- Bug follows mini Implementation→Testing cycle -- On bug closure, return to main testing discussion -- Bug results integrated into main test checklist - -_The bug sub-cycle mirrors Stages 4–6 (Implementation → Testing → Review) and inherits the same promotion and voting policies._ - -#### Automated Actions on commit -- Appends AI comment with vote to discussion.md - - Moderates discussion - - establishes fix objectives from plan.md - - delegates fix tasks from tasks.md -- Maintains discussions/discussion.sum.md -- Creates/maintains fix/src/* files if any are produced during the discussion. - -#### Promotion Actions -- Creates or report to discussions/review.discussion.md (OPEN) -- Creates review/findings_BUG_YYYYMMDD_.md with verification summary - -### Stage 6: Review Discussion -- File: discussions/review.discussion.md - -#### Header - -```yaml ---- -type: discussion -stage: review -status: OPEN # OPEN | READY_FOR_RELEASE | CHANGES_REQUESTED -feature_id: FR_YYYY-MM-DD_ -stage_id: FR_YYYY-MM-DD__review -promotion_rule: - allow_agent_votes: true - ready_min_eligible_votes: 1_human # HUMAN GATE - reject_min_eligible_votes: all -# ... ---- -``` - -#### Operational Flow -- AI_Reviewer summarizes into review/findings.md -- Review covers: changes, risks, test evidence, deployment considerations -- Can spawn follow-up feature requests or bugs from findings -- When human READY present and no blockers: status → READY_FOR_RELEASE - -_Promotion decisions obey thresholds in `process/policies.yml` (`voting.quorum`). The orchestrator re-evaluates eligibility before any status change._ - -#### Follow-up Artifact Creation -- New FR: ../../FR_YYYY-MM-DD_followup/request.md -- New Bug: bugs/BUG_YYYYMMDD_review/report.md - -### Stage 7: Release - -#### Entry Criteria -- Review discussion status is READY_FOR_RELEASE - -#### Automated Actions -- Generate release notes from feature changes -- Semver bump based on change type -- Create git tag -- Update changelog -- Document rollback procedure - -#### Post-Release -- Queue post-release validation tasks -- Update documentation as needed -- Archive feature folder if complete - -_State machine summary:_ All stage transitions are governed by the orchestrator and thresholds defined in `process/policies.yml`. Human gates remain mandatory for Implementation and Release. - - -## Voting, Quorum & Etiquette -### Voting System -Vote Values: READY | CHANGES | REJECT - -Format Requirements: -- Each comment must have a `Name:` line with the participant's name. -- The vote must be on a separate line, in the format: `VOTE: READY|CHANGES|REJECT` -- The vote should be at the end of the comment block. -- Multiple votes by same participant: latest wins. - -Vote Parsing Examples: - -```text -Name: Rob - -I agree with this approach. - -VOTE: READY -→ Rob: READY - -Name: AI_Claude - -Here's my analysis... - -VOTE: CHANGES -→ AI_Claude: CHANGES (if allow_agent_votes=true) - -Name: User - -I have concerns... - -VOTE: CHANGES - -Later: -Name: User - -Actually, addressed now. - -VOTE: READY -→ User: READY (latest vote wins) -``` -**Update Rules** -- Latest vote per participant supersedes all prior votes in the same stage. -- If a comment has multiple “VOTE:” lines, only the **last** valid line counts. -- Empty or malformed vote lines are ignored (no implicit abstain). - -### Eligibility & Quorum -Default Policy (machine-readable in process/policies.yml): - -```yaml -version: 1 -voting: - values: [READY, CHANGES, REJECT] - allow_agent_votes: true - quorum: - discussion: { ready: all, reject: all } - implementation: { ready: 1_human, reject: all } - release: { ready: 1_human, reject: all } -eligibility: - agents_allowed: true - require_human_for: [implementation, release] -etiquette: - name_prefix_agents: "AI_" - vote_line_regex: "^VOTE:\\s*(READY|CHANGES|REJECT)\\s*$" -timeouts: - discussion_stale_days: 3 - nudge_interval_hours: 24 -``` -Human Safety Gates: -- Implementation promotion: ≥1 human READY required -- Release promotion: ≥1 human READY required -- Agent votes count toward discussion but cannot satisfy human requirements - -### Promotion Evaluation Algorithm -1. Parse latest vote per participant (apply `vote_line_regex` to final non-empty line). -2. Filter by eligibility (humans/agents per stage policy). -3. Check human-gate requirement (if configured for the stage). -4. Evaluate quorum thresholds from `voting.quorum[stage]`: - - `ready: all` → all eligible voters are `READY` (or no `CHANGES/REJECT`) - - `ready: 1_human` → at least one human `READY` - - `reject: all` → if any `REJECT`, fail promotion -5. If thresholds met → flip `status` to the target (e.g., `READY_FOR_DESIGN`) **within the same commit** and generate next artifacts. -6. If not met → append moderator summary and keep status unchanged. - -Eligibility Definition -- Eligible voter: any participant (human or agent) who posted in the current stage discussion and conforms to eligibility.* policy. -- Human gate: stages listed in eligibility.require_human_for require ≥ 1 human READY regardless of agent votes. - -### Participation Etiquette -- Conciseness: Keep comments action-oriented and focused -- References: Link to files/sections when possible (design.md#architecture) -- Naming: Agents must prefix with AI_ (e.g., AI_Architect) -- Ownership: Suggest explicit owners for next steps (@AI_Architect: please draft...) -- Timeliness: Respond to direct questions within 24 hours -- Staleness & Nudges: If a stage has no new comments within `discussion_stale_days`, the AI_Moderator posts a nudge every `nudge_interval_hours` with missing votes and open questions. - -### Tie-breaks & Deadlocks -- If votes include both `READY` and `CHANGES/REJECT` beyond the promotion timeout (`promotion_timeout_days`), the AI_Moderator escalates: - 1) Summarize blocking points and owners, - 2) Request explicit human decision, - 3) If still unresolved after one more nudge window, maintain status and open a follow-up item in the summary’s **ACTION_ITEMS**. - -## Cascading Rules System -The Cascading Rules System defines how automation instructions are discovered and applied -for any file committed in the repository. The nearest `.ai-rules.yml` file to a changed -file determines how it will be processed. Rules can exist at three scopes: - -| Scope | Typical Path | Purpose | -|:------|:-------------|:---------| -| **Global** | `/.ai-rules.yml` | Default behavior for all files (e.g., code, diagrams) | -| **Feature-Scoped** | `Docs/features/.ai-rules.yml` | Rules specific to feature discussions and artifacts | -| **Local / Experimental** | `/local/.ai-rules.yml` (optional) | Overrides for prototypes or nested modules | - -Rule lookup always starts in the source file’s directory and walks upward until it finds -a `.ai-rules.yml`, then merges settings from outer scopes, where **nearest directory wins**. - -### Global Rules (Root .ai-rules.yml) -```yaml -version: 1 - -# Map file extensions to rule names -file_associations: - "*.js": "js-file" - "*.ts": "js-file" - "*.puml": "puml-file" - "*.md": "md-file" - -rules: - js-file: - description: "Generate PlantUML + review for JS/TS files" - outputs: - diagram: - enabled: true - path: "Docs/diagrams/file_diagrams/{basename}.puml" - output_type: "puml-file" - instruction: | - Update the PlantUML diagram to reflect staged code changes. - Focus on: key functions, control flow, data transformations, dependencies. - Keep architectural elements clear and focused. - review: - enabled: true - path: "Docs/discussions/reviews/{date}_{basename}.md" - output_type: "md-file" - instruction: | - Create technical review of code changes. - Include: summary of changes, potential risks, edge cases, - testing considerations, performance implications. - Use concise bullet points. - - puml-file: - description: "Rules for PlantUML diagram files" - instruction: | - Maintain readable, consistent diagrams. - Use descriptive element names, consistent arrow styles. - Include brief legend for complex diagrams. - - md-file: - description: "Rules for Markdown documentation" - instruction: | - Use proper Markdown syntax with concise paragraphs. - Use code fences for examples, lists for multiple points. - Maintain technical, clear tone. - -settings: - max_tokens: 4000 - temperature: 0.1 - model: "claude-sonnet-4-5-20250929" -``` -### Validation & Schema -Each `.ai-rules.yml` must pass a lightweight YAML schema check before execution: -- `version` key is required (integer or semver) -- `file_associations` maps glob patterns → rule names -- Each rule under `rules:` must include at least one of: - - `description`, `outputs:` with `path`, `output_type`, and `instruction` -- Unknown keys are ignored but logged as warnings. - -Schema validation prevents mis-typed keys from silently breaking automation. - -### Feature-Scoped Rules (Docs/features/.ai-rules.yml) -```yaml -version: 1 - -file_associations: - "request.md": "feature_request" - - # discussions - "feature.discussion.md": "feature_discussion" - "design.discussion.md": "design_discussion" - "implementation.discussion.md": "impl_discussion" - "testing.discussion.md": "test_discussion" - "review.discussion.md": "review_discussion" - - # summaries (companions) - "feature.discussion.sum.md": "discussion_summary" - "design.discussion.sum.md": "discussion_summary" - "implementation.discussion.sum.md": "discussion_summary" - "testing.discussion.sum.md": "discussion_summary" - "review.discussion.sum.md": "discussion_summary" - -rules: - feature_request: - outputs: - feature_discussion: - path: "{dir}/discussions/feature.discussion.md" - output_type: "feature_discussion_writer" - instruction: | - If missing: create with standard header (stage: feature, status: OPEN), - add Summary and Participation sections, then append initial AI comment with vote. - If exists: no op. - - # Also create the companion summary file if missing (blank sections with markers) - feature_summary_init: - path: "{dir}/discussions/feature.discussion.sum.md" - output_type: "discussion_summary_init" - instruction: | - If missing, create companion summary with stable markers: - DECISIONS, OPEN_QUESTIONS, AWAITING, ACTION_ITEMS, VOTES, TIMELINE, LINKS. - If exists, do not modify. - - feature_discussion: - outputs: - # 1) Append the new AI comment to the discussion (append-only) - self_append: - path: "{dir}/discussions/feature.discussion.md" - output_type: "feature_discussion_writer" - instruction: | - Append concise comment signed with AI name, ending with a single vote line. - Evaluate votes against header thresholds. If READY threshold met: - - Flip status to READY_FOR_DESIGN (or FEATURE_REJECTED) - Clearly state promotion decision. Append-only with minimal diff. - - # 2) Update the companion summary (marker-bounded sections only) - summary_companion: - path: "{dir}/discussions/feature.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Create or update the summary file. Replace ONLY content between markers: - DECISIONS, OPEN_QUESTIONS, AWAITING, ACTION_ITEMS, VOTES, TIMELINE, LINKS. - Inputs: the entire feature.discussion.md and current header. - Keep diffs minimal. - - # 3) Promotion artifacts when READY_FOR_DESIGN - design_discussion: - path: "{dir}/discussions/design.discussion.md" - output_type: "design_discussion_writer" - instruction: | - Create ONLY if feature discussion status is READY_FOR_DESIGN. - Seed with standard header (stage: design, status: OPEN). - - design_doc: - path: "{dir}/design/design.md" - output_type: "design_doc_writer" - instruction: | - Create ONLY if feature discussion status is READY_FOR_DESIGN. - Seed from request.md and feature discussion. - Include: Context, Options, Decision, Risks, Acceptance Criteria. - - # Ensure design summary exists once design discussion begins - design_summary_init: - path: "{dir}/discussions/design.discussion.sum.md" - output_type: "discussion_summary_init" - instruction: | - If missing, create companion summary with standard markers. - If exists, do not modify unless via discussion_summary_writer. - - design_discussion: - outputs: - design_update: - path: "{dir}/design/design.md" - output_type: "design_doc_writer" - instruction: | - Update design document to reflect latest design discussion. - Ensure acceptance criteria are measurable and complete. - Maintain all standard sections. Minimal diffs. - - # Always keep the design summary in sync (marker-bounded) - summary_companion: - path: "{dir}/discussions/design.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Update only the marker-bounded sections from the discussion content. - - impl_discussion: - path: "{dir}/discussions/implementation.discussion.md" - output_type: "impl_discussion_writer" - instruction: | - Create ONLY if design discussion status is READY_FOR_IMPLEMENTATION. - - impl_plan: - path: "{dir}/implementation/plan.md" - output_type: "impl_plan_writer" - instruction: | - Create ONLY if design status is READY_FOR_IMPLEMENTATION. - Draft implementation milestones and scope. - - impl_tasks: - path: "{dir}/implementation/tasks.md" - output_type: "impl_tasks_writer" - instruction: | - Create ONLY if design status is READY_FOR_IMPLEMENTATION. - Generate task checklist aligned to acceptance criteria. - - # Ensure implementation summary exists at the moment implementation starts - impl_summary_init: - path: "{dir}/discussions/implementation.discussion.sum.md" - output_type: "discussion_summary_init" - instruction: | - If missing, create companion summary with standard markers. - - impl_discussion: - outputs: - tasks_sync: - path: "{dir}/implementation/tasks.md" - output_type: "impl_tasks_maintainer" - instruction: | - Parse checkboxes and PR mentions from implementation discussion. - Synchronize tasks.md accordingly. - When all required tasks complete, mark implementation discussion READY_FOR_TESTING. - - summary_companion: - path: "{dir}/discussions/implementation.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Update only the marker-bounded sections from the discussion content. - Include unchecked items from ../implementation/tasks.md in ACTION_ITEMS. - - test_discussion: - path: "{dir}/discussions/testing.discussion.md" - output_type: "test_discussion_writer" - instruction: | - Create ONLY if implementation status is READY_FOR_TESTING. - - test_plan: - path: "{dir}/testing/testplan.md" - output_type: "testplan_writer" - instruction: | - Create ONLY if implementation status is READY_FOR_TESTING. - Derive strategy from acceptance criteria. - - test_checklist: - path: "{dir}/testing/checklist.md" - output_type: "testchecklist_writer" - instruction: | - Create ONLY if implementation status is READY_FOR_TESTING. - Generate test checklist covering acceptance criteria and edge cases. - - test_summary_init: - path: "{dir}/discussions/testing.discussion.sum.md" - output_type: "discussion_summary_init" - instruction: | - If missing, create companion summary with standard markers. - - test_discussion: - outputs: - checklist_update: - path: "{dir}/testing/checklist.md" - output_type: "testchecklist_maintainer" - instruction: | - Parse [RESULT] PASS/FAIL blocks from test discussion. - Update checklist accordingly with evidence links. - On test failure, create appropriate bug report. - - summary_companion: - path: "{dir}/discussions/testing.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Update marker-bounded sections from the discussion content. - Surface FAILs in OPEN_QUESTIONS or AWAITING with owners. - - bug_report: - path: "{dir}/bugs/BUG_{date}_auto/report.md" - output_type: "bug_report_writer" - instruction: | - Create bug report ONLY when test failure has clear reproduction steps. - Initialize bug discussion and fix plan in the same folder. - - review_discussion: - path: "{dir}/discussions/review.discussion.md" - output_type: "review_discussion_writer" - instruction: | - Create ONLY if all test checklist items pass. - Set testing discussion status to READY_FOR_REVIEW. - - review_findings: - path: "{dir}/review/findings.md" - output_type: "review_findings_writer" - instruction: | - Create summary of verified functionality, risks, and noteworthy changes. - - review_summary_init: - path: "{dir}/discussions/review.discussion.sum.md" - output_type: "discussion_summary_init" - instruction: | - If missing, create companion summary with standard markers. - - review_discussion: - outputs: - summary_companion: - path: "{dir}/discussions/review.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Update marker-bounded sections from the discussion content. - Decisions should include READY_FOR_RELEASE with date and follow-ups. - - followup_feature: - path: "../../FR_{date}_followup/request.md" - output_type: "feature_request_writer" - instruction: | - Create follow-up feature request ONLY when review identifies an enhancement. - - followup_bug: - path: "{dir}/bugs/BUG_{date}_review/report.md" - output_type: "bug_report_writer" - instruction: | - Create bug report ONLY when review identifies a defect. - Seed discussion and fix plan. - - # Generic writer invoked when a *.discussion.sum.md file itself is staged/edited - discussion_summary: - outputs: - normalize: - path: "{dir}/{basename}.sum.md" - output_type: "discussion_summary_normalizer" - instruction: | - Ensure standard header exists and marker blocks are present. - Do not rewrite content outside markers. - -``` - -> The shipped defaults focus on the feature → implementation flow; downstream stages (design, testing, review) reuse the same pattern and can be enabled by extending `.ai-rules.yml` inside the generated project. - -5.3 Rule Resolution Precedence -- Nearest Directory: Check source file directory and parents upward -- Feature Scope: Docs/features/.ai-rules.yml for feature artifacts -- Global Fallback: Root .ai-rules.yml for code files -- Conflict Resolution: Nearest rule wins, with logging of override decisions - -## Orchestration Architecture -**Principles** -- **Single-commit boundary:** automation only stages changes within the *current* commit; it never creates new commits or loops. -- **Deterministic prompts:** identical inputs produce identical patches (prompt hashing + stable sorting of inputs). -- **Nearest-rule wins:** rule resolution favors the closest `.ai-rules.yml`. -- **Fail fast, explain:** on any failure, keep the index untouched and write actionable diagnostics to `.git/ai-rules-debug/`. - -### Bash Pre-commit Hook -Core Responsibilities: -- Collect staged files (Added/Modified only) -- Resolve rules via cascading lookup -- Build context prompts from staged content -- Call AI model via CLI for patch generation -- Apply patches with robust error handling - -Prompt Envelope (deterministic) -```text -BEGIN ENVELOPE -VERSION: 1 -SOURCE_FILE: -RULE: / -FEATURE_ID: -STAGE: -POLICY_SHA256: -CONTEXT_FILES: -PROMPT_SHA256: ---- INPUT:FILE --- - ---- INPUT:POLICY --- - ---- INSTRUCTION --- - -END ENVELOPE -``` -On output, the model must return only a unified diff between -`<<>>` and `<<>>`. The orchestrator records -`PROMPT_SHA256` alongside the patch for reproducibility. - -Execution Order (per staged file) -1) **resolve_rules(rel_path)** → pick nearest `.ai-rules.yml`, match `file_associations`, assemble outputs. -2) **build_prompt(ctx)** → gather file content/diff, parsed headers, policy, `{feature_id}/{stage}` and neighboring artifacts. -3) **invoke_model(prompt)** → receive a **unified diff** envelope (no raw text rewrites). -4) **sanitize_diff()** → enforce patch constraints (no path traversal, within repo, size limits). -5) **apply_patch()** → try 3-way apply, then strict apply; stage only on success. -6) **log_diagnostics()** → write `resolution.log`, raw/clean/sanitized/final diffs. - -Enhanced Template Support: - -```bash -# Add/extend in resolve_template() function -resolve_template() { - local tmpl="$1" rel_path="$2" - local today dirpath basename name ext feature_id stage - today="$(date +%F)" - dirpath="$(dirname "$rel_path")" - basename="$(basename "$rel_path")" - name="${basename%.*}" - ext="${basename##*.}" - # nearest FR_* ancestor as feature_id - feature_id="$(echo "$rel_path" | sed -n 's|.*Docs/features/\(FR_[^/]*\).*|\1|p')" - # infer stage from .feature.discussion.md when applicable - stage="$(echo "$basename" | sed -n 's/^\([A-Za-z0-9_-]\+\)\.discussion\.md$/\1/p')" - echo "$tmpl" \ - | sed -e "s|{date}|$today|g" \ - -e "s|{rel}|$rel_path|g" \ - -e "s|{dir}|$dirpath|g" \ - -e "s|{basename}|$basename|g" \ - -e "s|{name}|$name|g" \ - -e "s|{ext}|$ext|g" \ - -e "s|{feature_id}|$feature_id|g" \ - -e "s|{stage}|$stage|g" -} -``` -Patch Application Strategy: -- Preserve Index Lines: Enable 3-way merge capability -- Try 3-way First: git apply --index --3way --recount --whitespace=nowarn -- Fallback to Strict: git apply --index if 3-way fails -- Debug Artifacts: Save raw/clean/sanitized/final patches to .git/ai-rules-debug/ - -Additional Safeguards: -- Reject patches that: - - create or edit files outside the repo root - - exceed 200 KB per artifact (configurable) - - modify binary or non-targeted files for the current output -- Normalize line endings; ensure new files include headers when required. -- Abort on conflicting hunks; do not partially apply a file’s patch. - -Discussion File Optimization: -- Prefer append-only edits with optional header flips -- For large files: generate full new content and compute diff locally -- Minimize hunk drift through careful patch construction -- Enforce append-only: refuse hunks that modify prior lines except header keys explicitly allowed (`status`, timestamps, `feature_id`, `stage_id`). - -### Python Orchestrator (automation/workflow.py) -#### Phase 1 (Non-blocking Status): - -```python -#!/usr/bin/env python3 -import json, os, sys, subprocess, re -from pathlib import Path - -def main(): - changed_files = read_changed_files() - status_report = analyze_discussion_status(changed_files) - if status_report: - print("AI-Workflow Status Report") - print(json.dumps(status_report, indent=2)) - sys.exit(0) # Always non-blocking in v1 -``` -#### Core Functions: -- Vote Parsing: Parse discussion files, track latest votes per participant -- Threshold Evaluation: Compute eligibility and quorum status -- Status Reporting: JSON output of current discussion state -- Decision Hints: Suggest promotion based on policy rules - -#### Optimization Notes: -- Memoize `load_policy()` and `parse_front_matter()` with LRU caches. -- Reuse a single regex object for `vote_line_regex`. -- Avoid re-reading unchanged files by comparing `git hash-object` results. - -#### CLI (v1): -- workflow.py --status # print stage/vote status for staged files -- workflow.py --summarize # regenerate summary sections to stdout (no write) -- workflow.py --dry-run # run full pipeline but do not stage patches -- Outputs are written to stdout and .git/ai-rules-debug/orchestrator.log. - -#### Future Enhancements: -- Policy enforcement based on process/policies.yml -- Gitea API integration for issue/PR management -- Advanced agent coordination and task routing - -#### Model Invocation (env-configured) -- `AI_MODEL_CMD` (default: `claude`) and `AI_MODEL_OPTS` are read from env. -- `AI_RULES_SEED` allows deterministic sampling where supported. -- If the model returns non-diff output, the hook aborts with diagnostics. - -#### Environment toggles: -- `AI_RULES_MAX_JOBS` caps parallel workers (default 4) -- `AI_RULES_CACHE_DIR` overrides `.git/ai-rules-cache` -- `AI_RULES_DISABLE_CACHE=1` forces re-generation -- `AI_RULES_CI=1` enables dry-run & cache-only in CI -### Gitea Integration (Future) - -#### Label System: -- stage/*: stage/discussion, stage/design, stage/implementation, etc. -- blocked/*: blocked/needs-votes, blocked/needs-human -- needs/*: needs/design, needs/review, needs/tests - -#### Automated Actions: -- Open/label PRs for implementation transitions -- Post status summaries to PR threads -- Create tracking issues for feature implementation -- Report status checks to PRs - -#### Happy Path (single changed discussion file) -```text -git add Docs/features/FR_x/.../feature.discussion.md -└─ pre-commit - ├─ resolve_rules → feature_discussion + summary_companion - ├─ build_prompt (PROMPT_SHA256=…) - ├─ invoke_model → <<>>…<<>> - ├─ sanitize_diff + guards - ├─ apply_patch (3-way → strict) - └─ write logs under .git/ai-rules-debug/ -``` - -## Moderator Protocol -### AI_Moderator Responsibilities -#### Conversation Tracking: -- Monitor unanswered questions (>24 hours) -- Track missing votes from active participants -- Identify stale threads needing attention -- Flag direct mentions that need responses - -##### Signals & Triggers -- **Unanswered Qs:** any line ending with `?` or prefixed `Q:` with an `@owner` and no reply within `response_timeout_hours`. -- **Missing Votes:** participants who posted in the stage but whose last non-empty line does not match `vote_line_regex`. -- **Stale Discussion:** no new comments within `discussion_stale_days`. -- **Promotion Drift:** conflicting votes present beyond `promotion_timeout_days`. - -#### Progress Reporting: -- Compute current vote tallies and thresholds -- List participants who haven't voted recently -- Summarize promotion status and remaining requirements -- Highlight blocking issues or concerns - -##### Comment Template & Constraints: -- Max 10 lines, each ≤ 120 chars. -- Sections in order: **UNANSWERED**, **VOTES**, **ACTION ITEMS**, **STATUS**. -- Always end with `VOTE: CHANGES` (so it never promotes by itself). - -#### Task Allocation: -- Suggest explicit owners for pending tasks -- Example: "AI_Architect: please draft the acceptance criteria section" -- Example: "Rob: could you clarify the deployment timeline?" - -##### Escalation Path: -- If blockers persist past `promotion_timeout_days`, ping owners + maintainer. -- If still unresolved after one more nudge interval, create a follow-up entry in the summary’s **ACTION_ITEMS** with owner + due date. - -### Moderator Implementation -Rule Definition (in Docs/features/.ai-rules.yml): - -```yaml -discussion_moderator_nudge: - outputs: - self_append: - path: "{dir}/discussions/{stage}.feature.discussion.md" - output_type: "discussion_moderator_writer" - instruction: | - Act as AI_Moderator. Analyze the entire discussion and: - - UNANSWERED QUESTIONS: - - List any direct questions unanswered for >24 hours (mention @names) - - Flag questions that need clarification or follow-up - - VOTE STATUS: - - Current tally: READY: X, CHANGES: Y, REJECT: Z - - Missing votes from: [list of participants without recent votes] - - Promotion status: [based on header thresholds] - - ACTION ITEMS: - - Suggest specific next owners for pending tasks - - Propose concrete next steps with deadlines - - Keep comment under 10 lines. End with "VOTE: CHANGES". - Append-only; minimal diff; update nothing else. - UNANSWERED: list @owners for Qs > response_timeout_hours. - VOTES: READY=X, CHANGES=Y, REJECT=Z; Missing=[@a, @b] - ACTION: concrete next steps with @owner and a due date. - STATUS: promotion readiness per process/policies.yml (voting.quorum). - Constraints: ≤10 lines; ≤120 chars/line; append-only; end with: - VOTE: CHANGES -``` -Nudge Frequency: Controlled by nudge_interval_hours in policies -> **Automation boundary:** Moderator comments are appended within the current commit; no auto-commits are created. - -## Error Handling & Resilience - -### Safety Invariants -- **No auto-commits:** automation only stages changes in the current commit. -- **Atomic per-file:** a patch for a file applies all-or-nothing; no partial hunks. -- **Append-first for discussions:** prior lines are immutable except allowed header keys. -- **Inside repo only:** patches cannot create/modify files outside the repository root. -- **Deterministic retry:** identical inputs → identical patches (same prompt hash). - -### Common Failure Modes -Patch Application Issues: -- Symptom: Hunk drift on large files, merge conflicts -- Mitigation: 3-way apply with index preservation, append-only strategies -- Fallback: Local diff computation from full new content -- Exit code: 2 (apply failure); write `final.diff` and `apply.stderr` - -Model Output Problems: -- Symptom: Malformed diff, missing markers, invalid patch format -- Mitigation: Extract between markers, validate with git apply --check -- Fallback: Clear diagnostics with patch validation output -- Exit code: 3 (invalid diff); write `raw.out`, `clean.diff`, `sanitize.log` - -Tooling Dependencies: -- Symptom: Missing yq, claude, or other required tools -- Mitigation: Pre-flight checks with clear error messages -- Fallback: Graceful degradation with feature-specific disabling -- Exit code: 4 (missing dependency); write `preflight.log` - -Rule Conflicts: -- Symptom: Multiple rules matching same file with conflicting instructions -- Mitigation: Nearest-directory precedence with conflict logging -- Fallback: Global rule application with warning -- Exit code: 5 (rule resolution); write `resolution.log` - -Guardrail Violations: -- Symptom: Patch touches forbidden paths, exceeds size, or edits outside markers -- Mitigation: Reject patch, print exact guard name and offending path/line count -- Exit code: 6 (guardrail); write `guards.json` - -### Retry & Idempotency -- Re-run the same commit contents → identical `PROMPT_SHA256` and identical patch. -- To force a new generation, change *only* the source file content or the rule instruction. -- `--dry-run` prints the unified diff without staging; useful for CI and reproduction. - -### Recovery Procedures - -#### Quick Triage Map -| Failure | Where to Look | Fast Fix | -|---|---|---| -| Patch won’t apply | `.git/ai-rules-debug/*/apply.stderr` | Rebase or re-run after pulling; if discussion, ensure append-only | -| Invalid diff envelope | `raw.out`, `clean.diff`, `sanitize.log` | Check that model returned `<<>>`; shorten file context | -| Rule not found | `resolution.log` | Verify `file_associations` and `{stage}`/`{feature_id}` resolution | -| Guardrail breach | `guards.json` | Reduce patch size, keep edits within markers, or adjust config limit | -| Missing dependency | `preflight.log` | Install tool or disable rule until available | - -Manual Override: - -```bash -# Bypass hook for emergency edits -git commit --no-verify -m "Emergency fix: manually overriding discussion status" - -# Manually update discussion header -# type: discussion -> status: READY_FOR_IMPLEMENTATION -``` -Debug Artifacts: -- All patch variants saved to .git/ai-rules-debug/ -- Timestamped files: raw, clean, sanitized, final patches -- Commit-specific directories for correlation -- Rule resolution decisions saved to `.git/ai-rules-debug/resolution.log` including matched rule, output keys, and template-expanded paths. - -Rollback Strategy: -- All generated artifacts are staged separately -- Easy partial staging: git reset HEAD for specific artifacts -- Full reset: git reset HEAD~1 to undo entire commit with generations - -Regenerate Safely: -```bash - # See what would be generated without staging anything - automation/workflow.py --dry-run - # Apply only after inspection - git add -p -```` -Bypass & Minimal Patch: -```bash - # Temporarily bypass the hook for urgent hand-edits - git commit --no-verify -m "Hotfix: manual edit, will reconcile with rules later" -``` - -### Audit Trail - -#### Patch Sanitization & Guards (summary) -- Validate unified diff headers; reject non-diff content. -- Enforce append-only on discussions; allow header keys: status, feature_id, stage_id, timestamps. -- Enforce marker-bounded edits for *.discussion.sum.md. -- Limit per-artifact patch size (default 200 KB; configurable). -- Reject paths escaping repo root or targeting binaries. -- See Appendix B for the normative, full rule set. - -#### Execution Logging: -- All rule invocations logged with source→output mapping -- Patch application attempts and outcomes recorded -- Vote calculations and promotion decisions documented - -#### Debug Bundle: - -```bash -.git/ai-rules-debug/ -├─ 20251021-143022-12345-feature.discussion.md/ -│ ├─ raw.out # Raw model output -│ ├─ clean.diff # Extracted patch -│ ├─ sanitized.diff # After sanitization -│ └─ final.diff # Final applied patch -└─ execution.log # Chronological action log -``` -### Operator Checklist (1-minute) -1. git status → confirm only intended files are staged. -2. Open .git/ai-rules-debug/…/apply.stderr (if failed) or final.diff. -3. If discussion file: ensure your change is append-only. -4. Re-run automation/workflow.py --dry-run and compare diffs. -5. If still blocked, bypass with --no-verify, commit, and open a follow-up to reconcile. - -## Security & Secrets Management - -### Security Principles -- **No plaintext secrets in Git** — ever. -- **Scan before stage** — block secrets at pre-commit, not in CI. -- **Redact on write** — debug logs and prompts never store raw secrets. -- **Least scope** — env vars loaded only for the current process; not persisted. - -### Secret Protection -#### Never Commit: -- API keys, authentication tokens -- Personal identifying information -- Internal system credentials -- Private configuration data - -#### Secret Scanning & Blocking (pre-commit): -- Run lightweight detectors before rule execution; fail fast on matches. -- Suggested tools (any one is fine): `git-secrets`, `gitleaks`, or `trufflehog` (regex mode). -- Provide a repo-local config at `process/secrets.allowlist` to suppress false positives. - -#### Redaction Policy: -- If a candidate secret is detected in an input file, the hook **aborts**. -- If a secret appears only in model output or logs, it is **replaced** with `***REDACTED***` before writing artifacts. - -#### Inbound/Outbound Data Handling: -- **Inbound (source & discussions):** if a suspected secret is present, the hook blocks the commit and points to the line numbers. -- **Outbound (logs, envelopes, diffs):** redact values and include a `[REDACTED:]` tag to aid debugging without leakage. - -#### Environment Variables: - -```bash -# Current approach -export CLAUDE_API_KEY="your_key" -# Future .env approach (git-ignored) -# .env file loaded via python-dotenv in Python components -``` -.gitignore (additions): -``` -.env -.env.* -# .env.local, .env.prod, etc. -*.key -*.pem -*.p12 -secrets/*.json -secrets/*.yaml -``` -Provide non-sensitive examples as *.sample: -- .env.sample with placeholder keys -- automation/config.sample.yml showing structure without values - -Configuration Management: -- Keep sensitive endpoints in automation/config.yml -- Use environment variable substitution in configuration -- Validate no secrets in discussions, rules, or generated artifacts -- Substitution happens **in-memory** during prompt build; no expanded values are written back to disk. -- Maintain a denylist of key names that must never appear in artifacts: `API_KEY, ACCESS_TOKEN, SECRET, PASSWORD, PRIVATE_KEY`. - -### Access Control -Repository Security: -- Assume all repository contents are potentially exposed -- No sensitive business logic in prompt instructions -- Regular security reviews of rule definitions -- Guardrails: outputs cannot target paths outside repo root; writes to `secrets/` are blocked. - -Agent Permissions: -- Limit file system access to repository scope -- Validate output paths stay within repository -- Sanitize all file operations for path traversal -- Prompt Redaction: when building the model prompt, mask env-like values with `***REDACTED***` for any key matching the denylist or high-entropy detector. -- See **Appendix B: Diff Application Rules (Normative)** for the full list of path/size/marker guardrails enforced during patch application. - - -### Incident Response & Rotation -- If a secret is accidentally committed, immediately: - 1) Rotate the key at the provider, - 2) Purge it from Git history (e.g., `git filter-repo`), - 3) Invalidate caches and re-run the secret scanner. -- Track rotations in a private runbook (outside the repo). - -### Preflight Checks (hook) -- Verify required tools present: `git`, `python3`, `yq` (optional), chosen secret scanner. -- Run secret scanner against **staged** changes; on hit → exit 11. -- Validate `.ai-rules.yml` schema; on error → exit 12. -- Confirm patch guards (size/paths); violations → exit 13. -- Diagnostics: write to `.git/ai-rules-debug/preflight.log`. - -## Performance & Scale Considerations -### Optimization Strategies - -#### Deterministic Caching & Batching: -- **Prompt cache**: reuse model outputs when `PROMPT_SHA256` is identical. -- **Batch compatible files**: same rule/output pairs with small contexts can be grouped. -- **Stable ordering**: sort staged files + outputs before batching to keep results repeatable. -- **Cache location**: `.git/ai-rules-cache/` (keys by `PROMPT_SHA256` + rule/output). - -#### Prompt Efficiency: -- Pass staged diffs instead of full file contents when possible -- Use concise, structured instructions with clear formatting -- Limit context to relevant sections for large files -- Preload policy once per run; inject only relevant subsections into the prompt -- Memoize parsed front-matter (YAML) and ASTs across files in the same run -- Trim discussion context to the last N lines (configurable) + stable summary - -#### Discussion Management: -- Append-only edits with periodic summarization -- Compact status reporting in moderator comments -- Archive completed discussions if they become too large -- Sliding-window summarization: regenerate `{stage}.discussion.sum.md` when diff > threshold lines -- Limit TIMELINE to the last 15 entries (configurable) - -#### Batch Operations: -- Process multiple related files in single model calls when beneficial -- Cache rule resolutions for multiple files in same directory -- Parallelize independent output generations -- Cap parallelism with `AI_RULES_MAX_JOBS` (default 4) to avoid CPU thrash. -- Deduplicate prompts for identical contexts across multiple outputs. - -### Scaling Limits - -#### File Size Considerations: -- Small (<100KB): Full content in prompts -- Medium (100KB-1MB): Diff-only with strategic context -- Large (>1MB): Chunked processing or summary-only approaches -- Very large (>5MB): refuse inline context; require pre-summarized artifacts - -#### Context Window Strategy: -- Hard cap prompt body at 200 KB per output (configurable) -- If over cap: (1) include diff; (2) include header + last 200 lines; (3) link to file path - -#### AST/Diagram Work: -- Cache ASTs in `.git/ai-rules-cache/ast/` keyed by `:` -- Rate-limit diagram updates to once per file per commit (guard duplicate runs) - -#### Repository Size: -- Current approach suitable for medium-sized repositories -- For very large codebases: scope rules to specific directories -- Consider rule disabling for generated/binary assets - -#### Rate Limiting: -- Model API calls: implement throttling and retry logic -- Gitea API: respect rate limits with exponential backoff -- File operations: batch where possible to reduce I/O - -#### Performance Telemetry (optional) -- Write `.git/ai-rules-debug/perf.json` with per-output timings: - `{ resolve_ms, prompt_ms, model_ms, sanitize_ms, apply_ms, bytes_in, bytes_out }` -- Summarize totals at end of run for quick regressions spotting. - -## Testing Strategy - -## Goals -- Prove determinism (same inputs → same patch). -- Prove guardrails (append-only, marker-bounded, path/size limits). -- Prove promotion math (votes, quorum, human gates). -- Keep runs fast and hermetic (temp repo, mock clock, seeded RNG). - -## Testing Tiers -### Unit Tests (Python): -- Vote parsing and eligibility calculation -- Policy evaluation and quorum determination -- Rules resolution and conflict handling -- Template variable substitution - -### Integration Tests (Bash + Python): -- End-to-end rule → prompt → patch → apply cycle -- Discussion status transitions and promotion logic -- Error handling and recovery procedures -- Multi-file rule processing - -### Artifact Validation: -- PlantUML syntax checking: plantuml -checkonly -- Markdown structure validation -- Template completeness checks -- YAML syntax validation - -### Golden & Snapshot Tests: -- **Prompt Envelope Golden**: compare against `tests/gold/envelopes/.txt` -- **Diff Output Golden**: compare unified diffs in `tests/gold/diffs/.diff` -- **Summary Snapshot**: write `{stage}.discussion.sum.md` and compare against `tests/snapshots//.discussion.sum.md` (markers only) - -### Property-Based Tests: -- Using `hypothesis` to fuzz discussion comments; invariants: - - last non-empty line drives the vote - - regex `vote_line_regex` never matches malformed lines - - marker-bounded writer never edits outside markers - -### Mutation Tests (optional): -- Run `mutmut` on `automation/workflow.py` vote math and ensure tests fail when logic is mutated. - -### Test Architecture -```text -tests/ -├─ unit/ -│ ├─ test_votes.py -│ ├─ test_policies.py -│ ├─ test_rules_resolution.py -│ └─ test_template_variables.py -├─ integration/ -│ ├─ run.sh # Main test runner -│ ├─ lib.sh # Test utilities -│ ├─ fixtures/ -│ │ └─ repo_skeleton/ # Minimal test repository -│ │ ├─ .ai-rules.yml -│ │ ├─ Docs/features/.ai-rules.yml -│ │ └─ Docs/features/FR_test/ -│ │ ├─ request.md -│ │ └─ discussions/ -│ │ └─ data/ -│ │ ├─ bigfile.md # >1MB to trigger chunking -│ │ ├─ bad.diff # malformed diff for sanitizer tests -│ │ ├─ secrets.txt # simulated secrets for scanner tests -│ │ └─ envelopes/ # golden prompt envelopes -│ ├─ gold/ -│ │ ├─ envelopes/ -│ │ └─ diffs/ -│ └─ test_cases/ -│ ├─ test_feature_promotion.sh -│ ├─ test_design_generation.sh -│ ├─ test_bug_creation.sh -│ ├─ test_append_only_guard.sh -│ ├─ test_summary_snapshot.sh -│ ├─ test_secret_scanner_block.sh -│ ├─ test_ci_cache_only_mode.sh -│ ├─ test_moderator_nudge.sh -│ └─ test_rule_precedence.sh -├─ bin/ -│ └─ claude # Fake deterministic model -├─ snapshots/ -│ └─ FR_test_case/ -│ ├─ feature.discussion.sum.md -│ └─ design.discussion.sum.md -└─ README.md -``` - -### Hermetic Test Utilities -- Mock clock: set SOURCE_DATE_EPOCH to freeze {date} expansions. -- Temp repo: each test case creates a fresh TMP_REPO with isolated .git. -- Seeded RNG: set AI_RULES_SEED for deterministic model variants. -- Filesystem isolation: tests write only under TMPDIR and .git/ai-rules-*. - -### Fake Model Implementation -Purpose: Deterministic testing without external API dependencies - -Implementation (tests/bin/claude): - -```bash -#!/bin/bash -# Fake Claude CLI for testing -# Reads prompt envelope from stdin, outputs a unified diff or injected error. -# Controls: -# AI_FAKE_ERR=diff|apply|malformed (force error modes) -# AI_FAKE_SEED= (deterministic variant) -# AI_FAKE_MODE=discussion|design (which template to emit) - -set -euo pipefail -prompt="$(cat)" - -if [[ "${AI_FAKE_ERR:-}" == "malformed" ]]; then - echo "this is not a diff" - exit 0 -fi - -target_file=$(echo "$prompt" | awk '/^SOURCE_FILE:/ {print $2}') -if echo "$prompt" | grep -q "RULE: .*feature_discussion/self_append"; then -cat << 'EOF' - -<<>> -diff --git a/Docs/features/FR_test/discussions/feature.discussion.md b/Docs/features/FR_test/discussions/feature.discussion.md -index 1234567..890abcd 100644 ---- a/Docs/features/FR_test/discussions/feature.discussion.md -+++ b/Docs/features/FR_test/discussions/feature.discussion.md -@@ -15,3 +15,6 @@ voting: - - ## Summary - Test feature for validation - + - +## Participation - +AI_Test: This is a test comment. VOTE: READY -EOF -elif echo "$prompt" | grep -q "RULE: .*discussion_summary_writer"; then -cat << 'EOF' -<<>> -diff --git a/Docs/features/FR_test/discussions/feature.discussion.sum.md b/Docs/features/FR_test/discussions/feature.discussion.sum.md -index 1111111..2222222 100644 ---- a/Docs/features/FR_test/discussions/feature.discussion.sum.md -+++ b/Docs/features/FR_test/discussions/feature.discussion.sum.md -@@ -5,3 +5,4 @@ - - ## Votes (latest per participant) - READY: 1 • CHANGES: 0 • REJECT: 0 - - Rob - -<<>> -EOF -else - # Default patch for other file types - echo "<<>>" - echo "diff --git a/README.md b/README.md" - echo "index 0000000..0000001 100644" - echo "--- a/README.md" - echo "+++ b/README.md" - echo "@@ -0,0 +1,1 @@" - echo "+Generated by fake model" - echo "<<>>"fi -``` -### Integration Test Runner -#### Key Test Scenarios -- Feature Promotion: request.md → feature.discussion.md → READY_FOR_DESIGN -- Design Generation: design.discussion.md → design.md updates -- Bug Creation: test failure → auto bug report generation -- Error Recovery: Malformed patch → graceful failure with diagnostics -- Rule Conflicts: Multiple rule matches → nearest-directory resolution -- Append-Only Guard: attempt to edit earlier lines in discussion → reject -- Summary Snapshot: only markers mutate; outside text preserved -- Secret Scanner: staged secret blocks commit with exit 11 -- CI Cache-only: with AI_RULES_CI=1 and cache miss → exit 20 -- Moderator Nudge: comment ≤10 lines, ends with `VOTE: CHANGES` -- Rule Precedence: local overrides feature, feature overrides global - -#### Test Execution -```bash -# Run full test suite -cd tests/integration -./run.sh - -# Run specific test case -./test_cases/test_feature_promotion.sh -``` - -Makefile (optional) -```make -.PHONY: test unit integ lint -test: unit integ -unit: -- pytest -q tests/unit -integ: -- cd tests/integration && ./run.sh -lint: -- ruff check automation src || true -``` - -### Continuous Validation -#### Pre-commit Checks: -- PlantUML syntax validation for generated diagrams -- Markdown link validation -- YAML syntax checking for rule files -- Template variable validation - -#### Performance Benchmarks: -- Rule resolution time for typical commit -- Patch generation and application duration -- Memory usage during large file processing -- **CI Mode (`AI_RULES_CI=1`)**: - - Default to `--dry-run` and **cache-only** model lookups. - - On cache miss, print the missing `PROMPT_SHA256`, skip invocation, and exit 20. - - Use to keep CI fast and reproducible. - -#### Coverage Targets: -- ≥90% line coverage on `automation/workflow.py` vote/quorum logic -- ≥80% branch coverage on rule resolution and guards - -#### Success Criteria: -- All golden prompts/diffs stable across runs (no drift) -- Guardrail tests fail if append-only/marker or path checks are removed - -## Source Intelligence Automation (Auto-Review + Auto-Diagram) - -### Purpose -To keep technical documentation and diagrams in sync with evolving source code. On every staged change to src/**/*.js|ts|py, the automation layer: -- Analyzes the diff and AST to produce a concise review summary -- Extracts structure and updates a PlantUML diagram in Docs/diagrams/file_diagrams/ - -### A) Folder Layout -```text -src/ -├─ automation/ -│ ├─ __init__.py -│ ├─ analyzer.py # parses diffs, extracts structure & metrics -│ ├─ reviewer.py # writes review summaries (md) -│ ├─ diagrammer.py # emits PUML diagrams -│ └─ utils/ -│ ├─ git_tools.py # staged diff, blob lookup -│ ├─ code_parser.py # AST helpers (JS/TS/Python) -│ └─ plantuml_gen.py # renders PlantUML text -``` - -### B) Operational Flow (Triggered by Hook) -```text -┌────────────────────────────────────────────────────────┐ -│ pre-commit hook (bash) │ -│ └──> detect src/**/*.js|ts|py changes │ -│ ├─> call automation/analyzer.py --file │ -│ │ ├─ parse diff + AST │ -│ │ ├─ collect functions, classes, calls │ -│ │ └─ emit JSON summary │ -│ ├─> reviewer.py → Docs/discussions/reviews/ │ -│ └─> diagrammer.py → Docs/diagrams/file_diagrams/│ -└────────────────────────────────────────────────────────┘ -``` -Each stage emits a unified diff so the same patch-application rules (3-way apply, append-only) still apply. - -### C) Sample Rule (Root .ai-rules.yml) -```yaml -js-file: - description: "Generate PlantUML + review for JS/TS files" - outputs: - diagram: - path: "Docs/diagrams/file_diagrams/{name}.puml" - output_type: "puml-file" - instruction: | - Parse code structure and update a PlantUML diagram: - - Modules, classes, functions - - Control-flow edges between major functions - review: - path: "Docs/discussions/reviews/{date}_{name}.md" - output_type: "md-file" - instruction: | - Summarize this commit’s code changes: - - What changed and why - - Possible risks / performance / security notes - - Suggested tests or TODOs -``` -Similar rules exist for py-file, ts-file, etc. - -### D) Core Algorithms (pseudocode) -```python -# 1 analyzer.py - -def analyze_source(path): - diff = git_diff(path) - tree = parse_ast(path) - funcs, classes = extract_symbols(tree) - flows = extract_calls(tree) - metrics = compute_metrics(tree) - return { - "file": path, - "functions": funcs, - "classes": classes, - "flows": flows, - "metrics": metrics, - "diff_summary": summarize_diff(diff), - } - -# 2 diagrammer.py - -def generate_puml(analysis): - nodes = [*analysis["classes"], *analysis["functions"]] - edges = analysis["flows"] - puml = "@startuml\n" - for n in nodes: - puml += f"class {n}\n" - for a, b in edges: - puml += f"{a} --> {b}\n" - puml += "@enduml\n" - return puml - -# 3 reviewer.py - -def generate_review(analysis): - return f"""# Auto Review — {analysis['file']} - -## Summary -{analysis['diff_summary']} - -## Key Functions -{', '.join(analysis['functions'][:10])} - -## Potential Risks -- TODO: evaluate complexity or security implications - -## Suggested Tests -- Unit tests for new/modified functions -""" -``` - -### E) Outputs -- .puml → Docs/diagrams/file_diagrams/{basename}.puml (keeps architecture maps current) -- .md → Docs/discussions/reviews/{date}_{basename}.md (rolling code review history) -Each output follows 3-way apply / append-only rules; every commit leaves a diff trail in .git/ai-rules-debug/. - -### F) Integration with Orchestrator -```python -# automation/workflow.py (aggregation example) -if src_changed(): - from automation import analyzer, reviewer, diagrammer - for f in changed_src_files: - data = analyzer.analyze_source(f) - diagrammer.update_puml(data) - reviewer.update_review(data) -``` -Future versions can post summaries to the feature’s implementation discussion and link diagrams into design/design.md. - -### G) Testing the Source Automation Layer -- Unit: tests/unit/test_code_parser.py, tests/unit/test_puml_gen.py -- Integration: tests/integration/test_cases/test_auto_review.sh, test_auto_diagram.sh -- Fixtures: tests/integration/fixtures/repo_skeleton/src/ with fake commits to verify generation - -### H) Security & Performance Notes -- Sandbox analysis only — no execution of user code -- AST parsing limited to static structure -- Large files (>5k lines): partial summarization -- Output capped to ≤ 200 KB per artifact - -### I) Deliverables Added to Milestones -- M0 → create src/automation/ skeleton -- M1 → functional auto-review + auto-diagram for JS/TS files -- M2 → extend to Python + PlantUML cross-linking in design docs - -## Discussion Summaries (Companion Artifacts per Stage) - -### What it is -For every {stage}.discussion.md, maintain a sibling {stage}.discussion.sum.md. It is append-minimal with bounded section rewrites only (between stable markers). Contents: decisions, vote tallies, open questions, awaiting replies, action items, compact timeline. - -### Where it lives -```text -Docs/features/FR_.../ -└─ discussions/ - ├─ feature.discussion.md - ├─ feature.discussion.sum.md - ├─ design.discussion.md - ├─ design.discussion.sum.md - ├─ implementation.discussion.md - ├─ implementation.discussion.sum.md - ├─ testing.discussion.md - ├─ testing.discussion.sum.md - ├─ review.discussion.md - └─ review.discussion.sum.md -``` - -### Header (machine-readable) -```yaml ---- -type: discussion-summary -stage: feature # feature|design|implementation|testing|review -status: ACTIVE # ACTIVE|SNAPSHOT|ARCHIVED -source_discussion: feature.discussion.md -feature_id: FR_YYYY-MM-DD_ -updated: YYYY-MM-DDTHH:MM:SSZ -policy: - allow_agent_votes: true - require_human_for: [implementation, review] ---- -``` - -### Stable section markers (for tiny diffs) -```markdown -# Summary — - - -## Decisions (ADR-style) -- (none yet) - - - -## Open Questions -- (none yet) - - - -## Awaiting Replies -- (none yet) - - - -## Action Items -- (none yet) - - - -## Votes (latest per participant) -READY: 0 • CHANGES: 0 • REJECT: 0 -- (no votes yet) - - - -## Timeline (most recent first) -- : - - - -## Links -- Related PRs: – -- Commits: – -- Design/Plan: ../design/design.md - -``` - -### How it updates -Trigger: whenever {stage}.discussion.md is staged, the hook also updates/creates {stage}.sum.md. - -Deterministic logic: -- Votes: parse latest vote per participant (eligibility per policy) -- Decisions: if header status flips (e.g., READY_FOR_IMPLEMENTATION), append an ADR entry -- Open Questions: lines ending with ? or flagged Q: with @owner if present -- Awaiting Replies: mentions with no response from that participant within response_timeout_hours -- Action Items: unchecked tasks (- [ ]) with @owner remain tracked until checked -- Timeline: last N (default 15) comment one-liners with timestamp and name -- Links: auto-add PRs (#123), SHAs, and cross-file links - -Rotation / snapshots (optional): when discussion grows large or on schedule, write discussions/summaries/.md (status: SNAPSHOT) and keep {stage}.sum.md trimmed while retaining Decisions/Open Q/Actions/Votes. - -### Rules (additions in Docs/features/.ai-rules.yml) -```yaml -file_associations: - "feature.discussion.md": "feature_discussion" - "design.discussion.md": "design_discussion" - "implementation.discussion.md": "impl_discussion" - "testing.discussion.md": "test_discussion" - "review.discussion.md": "review_discussion" - -rules: - feature_discussion: - outputs: - summary_companion: - path: "{dir}/discussions/feature.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Create or update the summary file. Replace ONLY content between - these markers: DECISIONS, OPEN_QUESTIONS, AWAITING, ACTION_ITEMS, - VOTES, TIMELINE, LINKS. Do not touch other lines. - Inputs: the entire feature.discussion.md. - design_discussion: - outputs: - summary_companion: - path: "{dir}/discussions/design.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Same summary policy as feature.sum.md; also add link to ../design/design.md. - impl_discussion: - outputs: - summary_companion: - path: "{dir}/discussions/implementation.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Same summary policy; include unchecked items from ../implementation/tasks.md. - test_discussion: - outputs: - summary_companion: - path: "{dir}/discussions/testing.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Same summary policy; include failing test artifacts and ensure FAILs surface in Open Questions or Awaiting. - review_discussion: - outputs: - summary_companion: - path: "{dir}/discussions/review.discussion.sum.md" - output_type: "discussion_summary_writer" - instruction: | - Same summary policy; Decisions should note READY_FOR_RELEASE with date and follow-ups. -``` - -### Orchestrator support (nice-to-have) -Provide workflow.py --summarize to output regenerated sections for tests/CI. Track awaiting replies via timestamps per author; if absent, mark as awaiting. - -### Testing additions -- Unit: parsing of votes, questions, mentions, action items -- Integration: commit a discussion with constructs → verify summary sections updated and only marker-bounded hunks changed -- Failure: malformed discussion / huge file → generator still writes sections; timeline truncates; no crash - -### Why this helps -Newcomers can open {stage}.sum.md and immediately see the state. Humans keep talking in the discussion; the system curates the signal in the summary. Promotions are transparent via Decisions. Open loops are visible and assigned. - -## Implementation Plan -### Milestone M0: Process Foundation -Deliverables: -- setup_project.py (Initialize Cascading Development repo) -- process/design.md (this document) -- process/policies.md + process/policies.yml -- process/templates/ (all four core templates) -- automation/agents.yml (role mappings) -- src/automation/ skeleton (analyzer.py, reviewer.py, diagrammer.py, utils/*) - -Success Criteria: -- All process documentation in place -- Policy definitions machine-readable -- Templates provide clear starting points - -### Milestone M1: Orchestrator MVP + Hook Enhancements -Deliverables: -- automation/workflow.py (non-blocking status reporter) -- Bash hook: {dir} template variable support -- Bash hook: Index preservation for 3-way apply -- Bash hook: Append-only optimization for discussions -- Auto-review + auto-diagram operational for JS/TS via root rules (js-file) - -Success Criteria: -- Python orchestrator reports discussion status -- Template variables work for feature folder paths -- 3-way apply handles merge conflicts gracefully - -### Milestone M2: Stage Automation & Moderator -Deliverables: -- Enhanced Docs/features/.ai-rules.yml with stage rules -- AI_Moderator implementation via discussion rules -- Python orchestrator: policy-based decision hints -- Test suite for feature promotion flow -- Discussion summaries: rules (discussion_summary_writer) + tests - -Success Criteria: -- Feature requests auto-create discussions -- Discussions promote through stages based on votes -- Moderator provides useful conversation guidance - -### Milestone M3: Gitea Integration -Deliverables: -- automation/adapters/gitea_adapter.py -- Automated PR creation and labeling -- Status reporting to PR threads -- Issue tracking integration - -Success Criteria: -- Implementation stage auto-creates PRs -- Review status visible in PR discussions -- Labels reflect current stage and blockers - -### Milestone M4: Bash to Python Migration -Deliverables: -- Core rule resolution logic in Python -- Patch generation and application in Python -- Bash hook as thin wrapper calling Python -- Enhanced error handling and diagnostics - -Success Criteria: -- Maintains current functionality with better maintainability -- Improved error messages and recovery options -- Consistent behavior across all operations - -## Risks & Mitigations -### Technical Risks -Over-Automation Bypassing Humans: - -Risk: Critical decisions made without human oversight -- Mitigation: Human READY gates for Implementation and Release stages -- Control: Manual override capability for all automated promotions -- Patch Instability on Large Files: - -Risk: Hunk drift and merge conflicts in long discussions -- Mitigation: 3-way apply with index preservation, append-only strategies -- Fallback: Local diff computation from full content regeneration - -Tooling Dependency Management: - -Risk: Version conflicts or missing dependencies break system -- Mitigation: Pre-flight validation with clear error messages -- Recovery: Graceful degradation with feature flags - -Context Limit Exceeded: -- Risk: AI models cannot process very large discussions -- Mitigation: Structured summarization, chunked processing -- Alternative: Focus on recent changes with reference to history - -13.2 Process Risks -Vote Manipulation or Gaming: -- Risk: Participants exploit voting system for unwanted outcomes -- Mitigation: Clear etiquette policies, human override capability -- Oversight: Moderator monitoring for voting patterns - -Discussion Fragmentation: - -Risk: Conversations become scattered across multiple files -- Mitigation: Clear stage boundaries, cross-references between discussions -- Tooling: Search and navigation aids for related artifacts - -Agent Coordination Conflicts: -- Risk: Multiple agents making conflicting changes -- Mitigation: Clear role definitions, sequential processing -- Resolution: Human maintainer as final arbiter - -13.3 Adoption Risks -Learning Curve: - -Risk: New contributors struggle with system complexity -- Mitigation: Comprehensive documentation, template guidance -- Support: AI_Moderator provides onboarding assistance - -Process Overhead: -- Risk: System creates too much ceremony for small changes -- Mitigation: Configurable rule enabling/disabling -- Flexibility: Bypass options for trivial changes - -## Initial Setup & Bootstrapping -To streamline project onboarding and ensure every repository begins with a structured, traceable starting point, this system includes: -- a one-time setup script (`setup_cascadingdev.py`) that initializes the folder structure and installs the hook, -- a `create_feature.py` tool for creating feature requests (with or without Ramble), -- and a concise `USER_GUIDE.md` in the user project for daily guidance. - -### Steps Performed: -- Create the canonical folder structure under `Docs/` and seed the initial FR folder. -- Install the pre-commit hook and default configuration files. -- Copy `create_feature.py` and (optionally) `ramble.py` into the user project root. -- Optionally run Ramble to help collect the first feature; otherwise prompt via CLI. -- Generate the first Feature Request folder and the initial discussion + summary. - -Example Implementation -```python -#!/usr/bin/env python3 -""" -setup_project.py — Initialize AI–Human Collaboration repo -""" -import os, subprocess, datetime - -def run_ramble(): - """Launch Ramble dialog to collect initial feature request""" - subprocess.run(["python3", "ramble.py", "--prompt", "Describe your initial feature request"]) - -def main(): - today = datetime.date.today().isoformat() - feature_dir = f"Docs/features/FR_{today}_initial-feature-request" - os.makedirs(f"{feature_dir}/discussions", exist_ok=True) - print(f"Creating {feature_dir}/ ...") - - # Generate initial request file from template - request_md = os.path.join(feature_dir, "request.md") - if not os.path.exists(request_md): - with open(request_md, "w") as f: - f.write("# Feature Request: Initial Project Setup\n\n" - "**Intent:** Describe project goals and first milestone.\n" - "**Motivation / Problem:** Why this system is needed.\n" - "**Constraints / Non-Goals:** ...\n" - "**Open Questions:** ...\n") - - # Run Ramble dialog to fill in details interactively - print("Launching Ramble interactive prompt...") - run_ramble() - print("Setup complete — run create_feature.py to add more features.") - -if __name__ == "__main__": - main() -``` -### Rationale -This setup process ensures that every repository starts with a consistent structure and a human-authored origin document, created in a conversational way. It also guarantees that the automation and templates are initialized before any feature work begins. - -### 14 Template Evolution -#### 14.1 Versioning Strategy -Template Location as Version: -- Current templates always in process/templates/ -- Breaking changes require new feature request and migration plan -- Existing features use templates current at their creation - -**User Guide** -- The authoritative `USER_GUIDE.md` lives in CascadingDev’s source (`assets/templates/USER_GUIDE.md`) and is copied - into user projects (root) at install time. Update the source and rebuild the bundle to propagate changes. - -Migration Guidance: -- Document template changes in release notes -- Provide automated migration scripts for simple changes -- Flag features using deprecated templates - -#### 14.2 Core Templates -Feature Request Template (process/templates/feature_request.md): - -```markdown -# Feature Request: - -**Feature ID**: <FR_YYYY-MM-DD_slug> -**Intent**: <one paragraph describing purpose> -**Motivation / Problem**: <why this is needed now> -**Constraints / Non-Goals**: <bulleted list of limitations> -**Rough Proposal**: <short implementation outline> -**Open Questions**: <bulleted list of uncertainties> -**Meta**: Created: <date> • Author: <name> -Discussion Template (process/templates/discussion.md): -``` - -```markdown ---- -type: discussion -stage: <feature|design|implementation|testing|review> -status: OPEN -feature_id: <FR_...> -created: <YYYY-MM-DD> - -promotion_rule: - allow_agent_votes: true - ready_min_eligible_votes: all - reject_min_eligible_votes: all - -participation: - instructions: | - - Append your input at the end as: "YourName: your comment…" - - Every comment must end with a vote line: "VOTE: READY|CHANGES|REJECT" - - Agents/bots must prefix names with "AI_" - -voting: - values: [READY, CHANGES, REJECT] ---- -## Summary -2-4 sentence summary of current state - -## Participation -comments appended below -Design Document Template (process/templates/design_doc.md): -``` - -```markdown -# Design — <FR id / Title> - -## Context & Goals -## Non-Goals & Constraints -## Options Considered -## Decision & Rationale -## Architecture Diagram(s) -## Risks & Mitigations -## Acceptance Criteria (measurable) -``` -Implementation Plan Template (process/templates/implementation_plan.md): - -```markdown -# Implementation Plan — <FR id / Title> - -## Scope -## Milestones -- [ ] M0: Initial setup and scaffolding -- [ ] M1: Core functionality -- [ ] M2: Testing and refinement -## Tasks -- [ ] specific actionable task -## Test Strategy -## Done When -- <clear, verifiable completion criteria> -``` - -Discussion Summary Template (process/templates/summary.md): - -```markdown -# Summary — <Stage Title> -(Automatically maintained. Edit the discussion; this file will follow.) - -<!-- SUMMARY:DECISIONS START --> -## Decisions (ADR-style) -- (none yet) -<!-- SUMMARY:DECISIONS END --> - -<!-- SUMMARY:OPEN_QUESTIONS START --> -## Open Questions -- (none yet) -<!-- SUMMARY:OPEN_QUESTIONS END --> - -<!-- SUMMARY:AWAITING START --> -## Awaiting Replies -- (none yet) -<!-- SUMMARY:AWAITING END --> - -<!-- SUMMARY:ACTION_ITEMS START --> -## Action Items -- (none yet) -<!-- SUMMARY:ACTION_ITEMS END --> - -<!-- SUMMARY:VOTES START --> -## Votes (latest per participant) -READY: 0 • CHANGES: 0 • REJECT: 0 -- (no votes yet) -<!-- SUMMARY:VOTES END --> - -<!-- SUMMARY:TIMELINE START --> -## Timeline (most recent first) -- <timestamp> <name>: <one-liner> -<!-- SUMMARY:TIMELINE END --> - -<!-- SUMMARY:LINKS START --> -## Links -- Related PRs: – -- Commits: – -- Design/Plan: ../design/design.md -<!-- SUMMARY:LINKS END --> -``` - -## Roles & Agent Personas -### Human Roles -- Maintainer: - - Final approval authority for critical stages - - System configuration and rule definition - - Conflict resolution and manual overrides -- Contributor: - - Feature authorship and implementation - - Participation in discussions and voting - - Review and testing responsibilities - -### AI Agent Roles -Implemented as Python scripts under `agents/` and registered per rule via the `participants` list in `.ai-rules.yml`. Each agent imports the shared SDK in `src/cascadingdev/agent/` to load the staged discussion, call the configured AI provider, and append one guarded response. Tool-style personas (researcher, visualizer) can set `background: true` so their work happens asynchronously after the initiating commit; these service agents never cast votes and only supply reference material. - -Example persona configuration used to prime prompts: -```yaml -agents: - AI_Researcher: - role: research_specialist - stages: [request, discussion, design] - capabilities: [web_search, documentation_review, best_practices] - voting_weight: 0.5 - - AI_Architect: - role: software_architect - stages: [design, implementation] - capabilities: [system_design, tradeoff_analysis, diagram_generation] - voting_weight: 0.8 - - AI_Implementer: - role: senior_developer - stages: [implementation, review] - capabilities: [code_generation, refactoring, testing_strategy] - voting_weight: 0.7 - - AI_Reviewer: - role: quality_engineer - stages: [review, test] - capabilities: [code_review, risk_assessment, security_analysis] - voting_weight: 0.9 - - AI_Moderator: - role: discussion_moderator - stages: [discussion, design, review] - capabilities: [progress_tracking, question_routing, vote_monitoring] - voting_weight: 0.3 -``` - -#### Role-Specific Responsibilities -- AI_Researcher: - - Find prior art, RFCs, and reference implementations - - Research technical constraints and dependencies - - Identify potential risks and mitigation strategies -- AI_Architect: - - Translate requirements into technical design plan - - Create and maintain architecture diagrams - - Evaluate trade-offs and make design recommendations -- AI_Implementer: - - Propose code structure and implementation approaches - - Generate code snippets and refactoring suggestions - - Develop testing strategies and fixture plans -- AI_Reviewer: - - Conduct adversarial code and design review - - Identify edge cases and failure modes - - Assess security and performance implications -- AI_Moderator: - - Track discussion progress and participation - - Identify unanswered questions and missing votes - - Suggest next steps and task ownership - -## Glossary -- FR: Feature Request — the initial document proposing new functionality -- Gate: Promotion decision point between stages based on policy thresholds -- Append-Only: Edit strategy that only adds new content to end of file, with optional header updates -- 3-way Apply: Patch application technique using blob IDs to reconcile content drift -- Cascading Rules: Rule resolution system where nearest directory's rules override parent directories -- Stage-Per-Discussion: Organizational pattern with separate conversation files for each development phase -- Human Gate: Promotion requirement that cannot be satisfied by AI votes alone -- Bug Sub-Cycle: Mini feature lifecycle automatically created for test failures -- Template Variables: Placeholders ({basename}, {name}, {ext}, {date}, {rel}, {dir}, {feature_id}, {stage}) resolved in rule paths -- Vote Threshold: Minimum number or type of votes required for promotion -- Status Machine: Defined state transitions for discussion files (OPEN → READY_FOR_* → etc.) -- Orchestrator: Central coordination component managing rule execution and status tracking - -## Appendices -### Appendix A: Complete Policy Configuration -```yaml -# process/policies.yml -version: 1 -voting: - values: [READY, CHANGES, REJECT] - allow_agent_votes: true - quorum: - discussion: { ready: all, reject: all } - design: { ready: all, reject: all } - implementation: { ready: 1_human, reject: all } - testing: { ready: all, reject: all } - review: { ready: 1_human, reject: all } -eligibility: - agents_allowed: true - require_human_for: [implementation, review] -etiquette: - name_prefix_agents: "AI_" - vote_line_regex: "^VOTE:\\s*(READY|CHANGES|REJECT)\\s*$" - response_timeout_hours: 24 -timeouts: - discussion_stale_days: 3 - nudge_interval_hours: 24 - promotion_timeout_days: 14 -moderation: - max_lines: 10 - max_line_length: 120 -security: - scanners: - enabled: true - tool: gitleaks # or git-secrets, trufflehog - allowlist_file: process/secrets.allowlist - redaction: - apply_to: - - logs - - prompts - - patches - denylist_keys: - - API_KEY - - ACCESS_TOKEN - - SECRET - - PASSWORD - - PRIVATE_KEY - guards: - block_paths: - - secrets/ - max_patch_kb: 200 - forbid_binary_edits: true -performance: - max_jobs: 4 - prompt_kb_cap: 200 - discussion_timeline_limit: 15 - cache: - enabled: true - dir: .git/ai-rules-cache - batching: - enabled: true - max_batch: 4 - ast_cache: - enabled: true - dir: .git/ai-rules-cache/ast -``` - -### Appendix B: Diff Application Rules (Normative) -- Patch Sanitization Rules: - - Preserve index lines for 3-way merge capability - - Remove only fragile metadata (similarity, rename info) - - Keep file mode lines (new file mode, deleted file mode) - - Ensure proper header formatting for new files -- Application Order: - - Attempt 3-way apply with recount and whitespace ignore - - Fall back to strict apply if 3-way fails - - For new files: use -p0 patch level with header rewriting - - Validate patch with --check before application -- Error Handling: - - Save all patch variants for debugging - - Provide clear diagnostics on failure - - Suggest manual resolution steps when automated recovery fails - -### Appendix C: Complete Agent Definitions -```yaml -# automation/agents.yml -version: 1 -agent_defaults: - voting_weight: 0.5 - require_human_approval: false - -agents: - AI_Researcher: - role: research_specialist - description: "Finds prior art, documentation, and best practices" - stages: [request, discussion, design] - capabilities: - - web_search - - documentation_review - - best_practices_research - - risk_identification - voting_weight: 0.5 - - AI_Architect: - role: software_architect - description: "Designs system architecture and evaluates trade-offs" - stages: [design, implementation] - capabilities: - - system_design - - tradeoff_analysis - - diagram_generation - - technology_selection - voting_weight: 0.8 - - AI_Implementer: - role: senior_developer - description: "Implements features and creates testing strategies" - stages: [implementation, review] - capabilities: - - code_generation - - refactoring - - testing_strategy - - performance_optimization - voting_weight: 0.7 - - AI_Reviewer: - role: quality_engineer - description: "Conducts security, risk, and quality analysis" - stages: [review, test] - capabilities: - - code_review - - risk_assessment - - security_analysis - - quality_validation - voting_weight: 0.9 - - AI_Moderator: - role: discussion_moderator - description: "Tracks progress and ensures participation" - stages: [discussion, design, review] - capabilities: - - progress_tracking - - question_routing - - vote_monitoring - - task_allocation - voting_weight: 0.3 -``` - -### Final Implementation Note -This v2.0 design document incorporates all insights from our collaborative discussion, providing a comprehensive framework for AI-human development collaboration. The system balances automation with appropriate human oversight, maintains the lightweight Git-native philosophy, and provides clear escalation paths from feature conception through release. - -The stage-per-discussion model with automated artifact generation creates a self-documenting development process that scales from small features to large, complex implementations. The integrated bug sub-cycle ensures that quality issues are handled systematically without disrupting the main development flow. - -Implementation Priority: Begin with Milestone M0 (process foundation) and proceed sequentially through the implementation plan, validating each milestone before proceeding to the next. - -Document Version: 2.1 -Last Updated: 2025-10-22 -Status: READY_FOR_IMPLEMENTATION - -Build Reference: This document (v2.1) applies to CascadingDev installer version matching VERSION in the repository root. diff --git a/docs/INSTALL.md b/docs/INSTALL.md deleted file mode 100644 index b2b392d..0000000 --- a/docs/INSTALL.md +++ /dev/null @@ -1,28 +0,0 @@ -# CascadingDev Installer - -## Requirements -- Python 3.10+ and git -- (Optional) PySide6 for GUI (`pip install PySide6`) - -## Quick start -```bash - -python setup_cascadingdev.py --target /path/to/new-project -``` - -### Skip GUI -```bash -python setup_cascadingdev.py --target /path/to/new-project --no-ramble -``` - -> After installation, open `USER_GUIDE.md` in your new project for daily usage. - -## Rebuild & Run (for maintainers) -Rebuild the bundle every time you change assets/ or the installer: -```bash -python tools/build_installer.py -``` -Then run only the bundled copy: -```bash - python install/cascadingdev-*/setup_cascadingdev.py --target /path/to/new-project -``` \ No newline at end of file diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md deleted file mode 100644 index 58a78dd..0000000 --- a/docs/PROGRESS.md +++ /dev/null @@ -1,442 +0,0 @@ -# CascadingDev Implementation Progress - -**Last Updated:** 2025-11-03 -**Overall Completion:** ~57% (M0✅ M1✅ M2🚧 M3❌ M4✅) - ---- - -## 📊 Quick Status Overview - -| Milestone | Target | Status | Completion | -|-----------|--------|--------|------------| -| **M0: Process Foundation** | Foundation docs and setup | ✅ Complete | 100% | -| **M1: Orchestrator MVP** | Core automation working | ✅ Complete | 100% | -| **M2: Stage Automation** | All 7 stages + moderator | 🚧 In Progress | 40% | -| **M3: Gitea Integration** | PR/issue automation | ❌ Not Started | 0% | -| **M4: Python Migration** | Bash → Python hook | ✅ Complete | 100% | - -**Current Focus:** Starting Stage 4 (Implementation) - ---- - -## ✅ Milestone M0: Process Foundation (100%) - -**Goal:** Establish project structure, documentation, and templates - -### Core Documentation -- [x] `docs/DESIGN.md` - System architecture (3,018 lines) -- [x] `docs/AUTOMATION.md` - User-facing automation guide -- [x] `docs/INSTALL.md` - Installation instructions -- [x] `CLAUDE.md` - AI assistant guidance -- [x] `AGENTS.md` - Developer guidelines -- [x] `README.md` - Project overview -- [x] `VERSION` - Semantic versioning - -### Setup Infrastructure -- [x] `src/cascadingdev/setup_project.py` - Project installer (478 lines) -- [x] `src/cascadingdev/cli.py` - CLI commands (doctor, build, smoke, release, pack) -- [x] `tools/build_installer.py` - Bundle builder -- [x] `tools/bundle_smoke.py` - End-to-end installer test -- [x] `pyproject.toml` - Package configuration - -### Templates (6 files) -- [x] `assets/templates/feature_request.md` - Feature request template -- [x] `assets/templates/feature.discussion.md` - Feature discussion template -- [x] `assets/templates/feature.discussion.sum.md` - Summary template -- [x] `assets/templates/design.discussion.md` - Design discussion template -- [x] `assets/templates/design_doc.md` - Design document template (needs enhancement) -- [x] `assets/templates/USER_GUIDE.md` - User guide shipped to projects - -### Policy Files -- [x] `config/ai.yml` - AI provider configuration -- [x] `assets/templates/process/policies.yml` - Process policies template - -### Ramble GUI -- [x] `assets/runtime/ramble.py` - Feature capture GUI (PySide6/PyQt5) -- [x] `assets/runtime/create_feature.py` - CLI feature creation -- [x] Template META system - JSON metadata in HTML comments - ---- - -## ✅ Milestone M1: Orchestrator MVP + Hook Enhancements (100%) - -**Goal:** Python automation engine with cascading rules - -### Core Automation Modules (2,469 lines total) -- [x] `automation/config.py` (182 lines) - Rule loading, merging, path resolution -- [x] `automation/runner.py` (146 lines) - Rule evaluation, output generation -- [x] `automation/patcher.py` (720 lines) - AI patch generation and application -- [x] `automation/workflow.py` (533 lines) - Vote tracking, status reporting -- [x] `automation/summary.py` (301 lines) - Summary file formatting -- [x] `automation/agents.py` (438 lines) - AI agent integration -- [x] `automation/ai_config.py` (149 lines) - Multi-provider configuration -- [x] `src/cascadingdev/agent/` (sdk/providers/patcher) - Shared helpers for participant scripts - -### Pre-commit Hook -- [x] `assets/hooks/pre-commit` (192 lines bash) -- [x] Secret detection (regex patterns) -- [x] Append-only validation for discussions -- [x] Summary file template creation -- [x] Python module orchestration - -### Cascading Rules System -- [x] Hierarchical .ai-rules.yml loading (nearest file wins) -- [x] Template variable support (`{feature_id}`, `{dir}`, `{basename}`, etc.) -- [x] Path normalization and security (blocks `../` escapes) -- [x] Rule merging with override semantics -- [x] 10 rule types defined in `assets/templates/rules/features.ai-rules.yml` - -### Multi-Provider AI System -- [x] Three optimization levels (fast/default/quality) -- [x] Fallback chains (Claude → Codex → Gemini) -- [x] Model hint propagation (rule → runner → patcher) -- [x] Cost optimization via intelligent routing -- [x] Environment variable overrides -- [x] Claude subagent setup script (`tools/setup_claude_agents.sh`) -- [x] **AI normalization system** (agents.normalize_discussion()) - - [x] Natural conversation → structured JSON extraction - - [x] Fast model usage for cost-effective extraction - - [x] Simple fallback for explicit markers when AI unavailable - - [x] Two-tier architecture (AI primary, regex fallback) - -### Testing Infrastructure -- [x] `tests/test_workflow.py` - Workflow automation tests (vote parsing, participant agents, summaries) -- [x] `tests/test_patcher.py` - Patch generation tests -- [x] `tests/test_runner.py` - Rule evaluation tests -- [x] `tests/test_config.py` - Config loading tests -- [x] `tests/test_utils.py` - Utility tests -- [x] `tests/test_template_meta.py` - Template metadata tests -- [x] `tests/test_build.py` - Build system tests -- [x] **Total: 39 tests (pytest -q), 100% passing** - ---- - -## 🚧 Milestone M2: Stage Automation & Moderator (45%) - -**Goal:** Implement all 7 stages of the development lifecycle - -### Stage 1: Request (100% ✅) -- [x] Template: `assets/templates/feature_request.md` -- [x] Rule: `feature_request` in features.ai-rules.yml -- [x] Automation: Creates `feature.discussion.md` on commit -- [x] Tested: Working in production - -### Stage 2: Feature Discussion (100% ✅) -- [x] Template: `assets/templates/feature.discussion.md` -- [x] Summary template: `assets/templates/feature.discussion.sum.md` -- [x] Rules: `feature_discussion_update`, `feature_discussion_writer` -- [x] Automation: - - [x] Vote tracking (VOTE: READY/CHANGES/REJECT) - - [x] **AI normalization** for natural conversation (agents.normalize_discussion()) - - [x] Question extraction from natural language - - [x] Participant agent scripts (`agents/moderator.py`, `agents/visualizer.py`) invoked via `.ai-rules.yml` `participants` - - [x] Background participant support (`background: true`) so researcher/visualizer can run asynchronously without casting votes - - [x] Action item tracking from conversational text - - [x] Decision tracking with context understanding - - [x] @mention tracking - - [x] Timeline generation - - [x] Summary file updates - - [x] Simple fallback for explicit line-start markers (DECISION:, QUESTION:, ACTION:) -- [x] Gate creation: `design.discussion.md` when status = READY_FOR_DESIGN -- [x] Tested: `tests/test_workflow.py` covers moderator invocation, idempotency, and visualizer diagram generation plus production validation - -### Stage 3: Design Discussion (100% ✅) -- [x] Template: `assets/templates/design.discussion.md` -- [x] **Enhanced template:** `assets/templates/design_doc.md` with comprehensive ADR structure -- [x] Rules: `design_gate_writer`, `design_discussion_writer` -- [x] Automation: - - [x] Gate creation when feature status = READY_FOR_DESIGN - - [x] Discussion file generation - - [x] Design document template ready for AI generation -- [x] **End-to-end tests:** 4 comprehensive tests created (2 passing, 1 skipped, 1 specification) -- [x] Tested: Promotion logic validated via unit tests - -**Completed (2025-11-02):** -1. ✅ Enhanced `design_doc.md` template with 14-section ADR structure -2. ✅ Created end-to-end test suite (test_stage_promotion.py) -3. ✅ Validated vote counting and threshold logic -4. ✅ Documented AI vote exclusion behavior - -**Remaining Work (5% - Polish):** -- [x] Integrate status update into runner/workflow (currently manual) -- [x] Test design document AI generation in production - -### Stage 4: Implementation Discussion (75% 🚧) -- [x] Template: `implementation.discussion.md` -- [x] Template: `implementation/plan.md` -- [x] Template: `implementation/tasks.md` -- [x] Rules: `implementation_gate_writer` -- [x] Rule: `implementation_discussion_writer` -- [ ] Automation: - - [x] Gate creation when design status = READY_FOR_IMPLEMENTATION - - [x] Task checkbox tracking (parse `- [ ]` / `- [x]`) - - [ ] PR/commit linking - - [ ] Progress tracking roll-up -- [x] Human gate: Require ≥1 human READY vote + all tasks complete -- [x] Tests: Workflow + promotion coverage for implementation stage - -**Next Steps:** -1. Track PR/commit references and surface them in summaries -2. Feed implementation progress into higher-level reporting (burn-down, dashboards) - -### Stage 5: Testing Discussion (0% ❌) -- [ ] Template: `testing.discussion.md` **← MISSING** -- [ ] Template: `testing/testplan.md` **← MISSING** -- [ ] Template: `testing/checklist.md` **← MISSING** -- [ ] Rules: `testing_gate_writer` **← MISSING** -- [ ] Rule: `testing_discussion_writer` **← MISSING** -- [ ] Automation: - - [ ] Gate creation when implementation complete - - [ ] Test result tracking ([RESULT] PASS/FAIL) - - [ ] Checklist progress - - [ ] Bug linking -- [ ] Tests: None - -**Next Steps:** -1. Create testing templates -2. Design test result format -3. Implement testing rules -4. Add result parser to workflow.py - -### Stage 6: Review Discussion (0% ❌) -- [ ] Template: `review.discussion.md` **← MISSING** -- [ ] Template: `review/findings.md` **← MISSING** -- [ ] Rules: `review_gate_writer` **← MISSING** -- [ ] Rule: `review_discussion_writer` **← MISSING** -- [ ] Automation: - - [ ] Gate creation when testing complete - - [ ] Finding tracking - - [ ] Approval tracking -- [ ] Human gate: Require ≥1 human READY vote -- [ ] Tests: None - -**Next Steps:** -1. Create review templates -2. Design review findings format -3. Implement review rules -4. Test human gate enforcement - -### Stage 7: Release (0% ❌) -- [ ] Template: Changelog generation **← MISSING** -- [ ] Template: Rollback notes **← MISSING** -- [ ] Rules: `release_writer` **← MISSING** -- [ ] Automation: - - [ ] Changelog from commits - - [ ] Version tagging - - [ ] Release notes -- [ ] Human gate: Require maintainer approval -- [ ] Tests: None - -**Next Steps:** -1. Design release automation -2. Create changelog generator -3. Implement version tagging -4. Add maintainer role checking - -### AI_Moderator Protocol (0% ❌) -- [ ] Nudge system for inactive discussions -- [ ] Escalation paths for blocked features -- [ ] Conversation guidance -- [ ] Question tracking and follow-up -- [ ] Vote reminder system -- [ ] Timeout detection - -**Implementation Location:** `automation/moderator.py` (does not exist) - -**Next Steps:** -1. Create moderator.py module -2. Implement nudge timing logic -3. Add escalation rules to policies.yml -4. Integrate with workflow.py - -### Bug Sub-Cycles (0% ❌) -- [ ] BUG_YYYYMMDD_slug folder structure -- [ ] Bug-specific templates -- [ ] Bug tracking rules -- [ ] Integration with testing stage -- [ ] Bug lifecycle automation - -**Next Steps:** -1. Design bug folder structure -2. Create bug templates -3. Add bug rules to features.ai-rules.yml -4. Link bugs to parent features - -### Discussion Summaries (90% ✅) -- [x] Marker-based updates (SUMMARY:* blocks) -- [x] Vote tallies -- [x] Question tracking -- [x] Action items -- [x] Decisions -- [x] @mentions -- [x] Timeline -- [ ] Links (PR/commit auto-detection) - partially implemented -- [ ] Snapshots for large discussions - ---- - -## ❌ Milestone M3: Gitea Integration (0%) - -**Goal:** Integrate with Gitea for PR/issue automation - -### Gitea Adapter -- [ ] `automation/adapters/gitea_adapter.py` **← MISSING** -- [ ] Gitea API client integration -- [ ] Authentication setup - -### PR Automation -- [ ] Auto-create PRs from implementation stage -- [ ] Link PRs to feature discussions -- [ ] Update PR descriptions with status -- [ ] Auto-label PRs based on stage -- [ ] Comment with review findings - -### Issue Tracking -- [ ] Create issues from action items -- [ ] Link issues to discussions -- [ ] Update issue status from discussions -- [ ] Close issues when tasks complete - -### Status Reporting -- [ ] Post stage status to PR comments -- [ ] Update PR labels on stage changes -- [ ] Link to discussion summaries -- [ ] Report blocker status - -**Next Steps:** -1. Research Gitea API capabilities -2. Design adapter interface -3. Implement basic PR creation -4. Test with local Gitea instance - ---- - -## ✅ Milestone M4: Bash to Python Migration (100%) - -**Goal:** Migrate from bash-heavy hook to Python-powered automation - -### Architecture -- [x] Core rule resolution in Python (automation/config.py) -- [x] Patch generation in Python (automation/patcher.py) -- [x] Bash hook as thin wrapper (192 lines) -- [x] Python modules handle complex logic (2,469 lines) - -### Bash Hook Responsibilities (Minimal) -- [x] Secret detection (regex patterns) -- [x] Append-only validation -- [x] Summary file template creation -- [x] Python module orchestration - -### Python Module Responsibilities (Core) -- [x] Rule loading and cascading -- [x] AI prompt generation -- [x] Patch generation and application -- [x] Vote tracking and summary updates -- [x] Provider integration and fallback - -### Error Handling -- [x] Python exception handling -- [x] Graceful degradation -- [x] Debug logging (.git/ai-rules-debug/) -- [x] Clear error messages - -**Status:** Completed early in development (built right from the start) - ---- - -## 📈 Overall Progress Summary - -### By Component Type - -**Templates:** 6/13 (46%) -- ✅ Feature request, discussions (feature, design) -- ❌ Implementation, testing, review templates missing - -**Rules:** 10/16 (63%) -- ✅ Request, feature, design rules complete -- 🚧 Implementation gate defined but untested -- ❌ Testing, review, release rules missing - -**Automation Modules:** 7/9 (78%) -- ✅ Core modules complete (config, runner, patcher, workflow, summary, agents, ai_config) -- ❌ Moderator, Gitea adapter missing - -**Testing:** 18/30+ (60%) -- ✅ 18 tests passing (core automation) -- ❌ Stage promotion tests missing -- ❌ Integration tests needed - -**Documentation:** 7/7 (100%) -- ✅ All major docs complete and up-to-date - ---- - -## 🎯 Recommended Next Steps - -### Short Term (1-2 weeks) -1. **Complete Stage 3:** - - [ ] Enhance design_doc.md template - - [ ] Add end-to-end design stage test - - [ ] Document design stage workflow - -2. **Start Stage 4 (Implementation):** - - [ ] Create implementation templates (discussion, plan, tasks) - - [ ] Implement task checkbox parser - - [ ] Add human gate enforcement - - [ ] Test implementation stage promotion - -### Medium Term (3-4 weeks) -3. **Add Stage 5 (Testing):** - - [ ] Create testing templates - - [ ] Implement test result tracking - - [ ] Add checklist automation - -4. **Add Stage 6 (Review):** - - [ ] Create review templates - - [ ] Implement findings tracking - - [ ] Add human gate enforcement - -### Long Term (5-8 weeks) -5. **Complete Stage 7 (Release):** - - [ ] Design release automation - - [ ] Implement changelog generation - - [ ] Add version tagging - -6. **Implement AI_Moderator:** - - [ ] Create moderator.py module - - [ ] Add nudge system - - [ ] Implement escalation paths - -7. **Add Bug Sub-Cycles:** - - [ ] Design bug workflow - - [ ] Create bug templates - - [ ] Integrate with testing stage - -### Optional (Future) -8. **Gitea Integration (M3):** - - [ ] Research Gitea API - - [ ] Implement PR automation - - [ ] Add issue tracking - ---- - -## 📝 How to Update This Document - -When completing items: -1. Change `[ ]` to `[x]` for completed checkboxes -2. Update completion percentages in section headers -3. Update "Last Updated" timestamp at top -4. Update "Overall Completion" percentage -5. Update "Current Focus" line -6. Move items from "Next Steps" to checkboxes as work progresses -7. Commit changes: `git add docs/PROGRESS.md && git commit -m "docs: update progress tracking"` - ---- - -## 🔗 Related Documents - -- **DESIGN.md** - Full system architecture and design rationale -- **AUTOMATION.md** - User-facing automation guide -- **CLAUDE.md** - AI assistant context and guidance -- **AGENTS.md** - Developer guidelines and conventions -- **README.md** - Project overview and quick start diff --git a/docs/ai-provider-fallback.puml b/docs/ai-provider-fallback.puml deleted file mode 100644 index 65203a0..0000000 --- a/docs/ai-provider-fallback.puml +++ /dev/null @@ -1,163 +0,0 @@ -@startuml ai-provider-fallback -!theme plain -title AI Provider Fallback Chain with Model Hints - -start - -:Automation needs AI generation\n(from patcher.py or runner.py); - -:Read config/ai.yml; - -if (Rule has model_hint?) then (yes) - if (model_hint == "fast"?) then (yes) - :Use **command_chain_fast**: - - claude -p (→ Haiku subagent) - - codex --model gpt-5-mini - - gemini --model gemini-2.5-flash; - else if (model_hint == "quality"?) then (yes) - :Use **command_chain_quality**: - - claude -p (→ Sonnet subagent) - - codex --model o3 - - gemini --model gemini-2.5-pro; - else (unknown hint) - :Fall back to default chain; - endif -else (no hint) - :Use **command_chain** (default): - - claude -p (→ auto-select subagent) - - codex --model gpt-5 - - gemini --model gemini-2.5-flash; -endif - -partition "Provider Loop" { - :Get next provider from chain; - - if (Provider == "claude"?) then (yes) - :Execute: **claude -p**; - note right - Claude CLI uses TASK COMPLEXITY hint - from prompt to select subagent: - - FAST → cdev-patch (Haiku) - - QUALITY → cdev-patch-quality (Sonnet) - - Default → auto-select - end note - - if (Returned output?) then (yes) - if (Contains diff markers?) then (yes) - :✓ Success! Extract diff; - stop - else (no - non-diff response) - :Log: "Claude non-diff output"; - :Try next provider; - endif - else (command failed) - :Log: "Claude command failed"; - :Try next provider; - endif - - else if (Provider == "codex"?) then (yes) - :Execute: **codex exec --model X --json -**; - note right - Codex requires special handling: - - Add "exec" subcommand - - Add "--json" flag - - Add "--color=never" - - Add "-" to read from stdin - - Parse JSON output for agent_message - end note - - if (Exit code == 0?) then (yes) - :Parse JSON lines; - :Extract agent_message text; - - if (Contains diff?) then (yes) - :✓ Success! Extract diff; - stop - else (no diff) - :Log: "Codex no diff output"; - :Try next provider; - endif - else (exit code 1) - :Log: "Codex exited with 1"; - :Try next provider; - endif - - else if (Provider == "gemini"?) then (yes) - :Execute: **gemini --model X**; - note right - Gemini is the most reliable fallback: - - Accepts plain text input - - Returns consistent output - - Supports sentinel token - end note - - if (Returned output?) then (yes) - if (Output == sentinel token?) then (yes) - :Log: "No changes needed"; - :Return empty (intentional); - stop - else if (Contains diff?) then (yes) - :✓ Success! Extract diff; - stop - else (no diff) - :Log: "Gemini no diff output"; - :Try next provider; - endif - else (command failed) - :Log: "Gemini command failed"; - :Try next provider; - endif - endif - - if (More providers in chain?) then (yes) - :Continue loop; - else (no) - :✗ All providers failed; - :Raise PatchGenerationError; - stop - endif -} - -stop - -legend bottom - **Configuration Example (config/ai.yml):** - - runner: - command_chain: - - "claude -p" - - "codex --model gpt-5" - - "gemini --model gemini-2.5-flash" - - command_chain_fast: - - "claude -p" - - "codex --model gpt-5-mini" - - "gemini --model gemini-2.5-flash" - - command_chain_quality: - - "claude -p" - - "codex --model o3" - - "gemini --model gemini-2.5-pro" - - sentinel: "CASCADINGDEV_NO_CHANGES" - - **Environment Override:** - export CDEV_AI_COMMAND="claude -p || gemini --model gemini-2.5-pro" - (Overrides config.yml for this commit only) -endlegend - -note right - **Why Fallback Chain?** - - 1. **Redundancy**: Rate limits, API outages - 2. **Model specialization**: Different models excel at different tasks - 3. **Cost optimization**: Try cheaper models first - 4. **Quality assurance**: Fast models for simple tasks, quality for complex - - **Observed Behavior:** - - Claude occasionally returns non-diff output - - Codex consistently exits with code 1 (auth issues?) - - Gemini is the most reliable fallback -end note - -@enduml diff --git a/docs/architecture-overview.puml b/docs/architecture-overview.puml deleted file mode 100644 index ba3047b..0000000 --- a/docs/architecture-overview.puml +++ /dev/null @@ -1,66 +0,0 @@ -@startuml architecture-overview -!theme plain -title CascadingDev Architecture Overview - -package "User Project" { - folder "Docs/features/" { - file "request.md" as request - folder "discussions/" { - file "feature.discussion.md" as discussion - file "feature.discussion.sum.md" as summary - } - } - - folder ".git/hooks/" { - file "pre-commit" as hook - } - - folder "automation/" { - file "runner.py" as runner - file "config.py" as config - file "patcher.py" as patcher - file "workflow.py" as workflow - file "agents.py" as agents - file "summary.py" as summarymod - } - - folder "Docs/features/.ai-rules.yml" as rules -} - -cloud "AI Provider" { - component "Claude API" as claude - component "Claude CLI" as cli -} - -actor Developer - -Developer --> request: 1. Creates/edits -Developer --> hook: 2. git commit -hook --> runner: 3. Invokes -runner --> config: 4. Loads rules -config --> rules: 5. Reads -runner --> patcher: 6. Generate outputs -patcher --> claude: 7. AI request -claude --> patcher: 8. Returns patch -patcher --> discussion: 9. Applies patch -hook --> workflow: 10. Process votes -workflow --> agents: 11. Extract data -workflow --> summarymod: 12. Update summary -hook --> Developer: 13. Commit succeeds - -note right of runner - Orchestrates the AI - automation pipeline -end note - -note right of patcher - Generates and applies - AI-created patches -end note - -note right of workflow - Tracks votes and - updates summaries -end note - -@enduml diff --git a/docs/cascading-rules.puml b/docs/cascading-rules.puml deleted file mode 100644 index b023c62..0000000 --- a/docs/cascading-rules.puml +++ /dev/null @@ -1,76 +0,0 @@ -@startuml cascading-rules -!theme plain -title Cascading Rules Configuration System - -package "Repository Root" { - file ".ai-rules.yml" as root_rules #LightBlue - - package "Docs/features/" { - file ".ai-rules.yml" as features_rules #LightGreen - - package "FR_123/" { - file ".ai-rules.yml" as feature_rules #LightYellow - file "request.md" as request - - folder "discussions/" { - file "feature.discussion.md" as discussion - } - } - } -} - -component "config.py" as config - -request --> config: Process file -config --> feature_rules: 1. Load (nearest) -config --> features_rules: 2. Load (parent) -config --> root_rules: 3. Load (root) -config --> config: 4. Deep merge\n(nearest wins) - -note right of config - **Cascading Precedence:** - 1. Nearest directory rules - 2. Parent directory rules - 3. Root rules - - **Merge Strategy:** - - Nested dictionaries are merged recursively - - Arrays are replaced (not merged) - - Nearest values win on conflicts -end note - -note top of feature_rules - **FR_123/.ai-rules.yml** - - rules: - feature_request: - outputs: - feature_discussion: - # Override specific output config - instruction_append: | - Additional context for FR_123 -end note - -note top of features_rules - **Docs/features/.ai-rules.yml** - - file_associations: - "request.md": "feature_request" - "feature.discussion.md": "feature_discussion_update" - - rules: - feature_request: - outputs: - feature_discussion: - path: "Docs/features/{feature_id}/discussions/feature.discussion.md" - output_type: "feature_discussion_writer" -end note - -note top of root_rules - **Root .ai-rules.yml** - - Global rules that apply - to the entire repository -end note - -@enduml diff --git a/docs/commit-workflow.puml b/docs/commit-workflow.puml deleted file mode 100644 index d91b9ee..0000000 --- a/docs/commit-workflow.puml +++ /dev/null @@ -1,148 +0,0 @@ -@startuml commit-workflow -!theme plain -title Git Commit Workflow with Automation - -actor Developer -participant "git commit" as git -participant "pre-commit hook" as hook -participant "runner.py" as runner -participant "config.py" as config -participant "patcher.py" as patcher -participant "AI Providers" as ai -participant "Claude CLI" as claude -participant "Codex CLI" as codex -participant "Gemini CLI" as gemini -participant "workflow.py" as workflow -database ".git index" as index - -Developer -> git: commit staged files -activate git - -git -> hook: trigger pre-commit -activate hook - -hook -> runner: python3 -m automation.runner -activate runner - -runner -> config: Load .ai-rules.yml -activate config -config -> config: Find cascading rules -config -> runner: Return merged config -deactivate config - -loop For each staged file - runner -> config: Get rule for file - config -> runner: Return rule & outputs - - runner -> patcher: generate_output(source, target, instruction) - activate patcher - - patcher -> patcher: Build prompt with\nsource diff + context - - patcher -> ai: Try provider 1/3 - activate ai - ai -> claude: Send prompt - activate claude - - alt Claude produces diff - claude -> ai: Return unified diff\n(wrapped in markers) - ai -> patcher: Success with diff - deactivate claude - deactivate ai - patcher -> patcher: Extract & sanitize patch - patcher -> patcher: git apply --3way - patcher -> index: Stage generated file - patcher -> runner: Success - else Claude non-diff output - claude -> ai: Non-diff response - deactivate claude - ai -> codex: Try provider 2/3 - activate codex - - alt Codex succeeds - codex -> ai: Return diff (JSON parsed) - deactivate codex - deactivate ai - patcher -> patcher: Extract & sanitize - patcher -> index: Stage file - patcher -> runner: Success - else Codex fails (exit 1) - codex -> ai: Exit code 1 - deactivate codex - ai -> gemini: Try provider 3/3 - activate gemini - - alt Gemini succeeds - gemini -> ai: Return diff - deactivate gemini - deactivate ai - patcher -> patcher: Extract & sanitize - patcher -> index: Stage file - patcher -> runner: Success - else All providers failed - gemini -> ai: Error/no diff - deactivate gemini - deactivate ai - patcher -> patcher: Log error to stderr - patcher -> runner: Skip this file - end - end - end - - deactivate patcher -end - -runner -> hook: Exit 0 -deactivate runner - -hook -> workflow: python3 -m automation.workflow --status -activate workflow - -workflow -> workflow: Parse VOTE: lines\nfrom discussions -workflow -> workflow: Update .sum.md files -workflow -> index: Stage updated summaries -workflow -> hook: Exit 0 -deactivate workflow - -hook -> git: Exit 0 (continue commit) -deactivate hook - -git -> index: Create commit -git -> Developer: Commit successful -deactivate git - -note right of ai - **Multi-Provider Fallback Chain** - Configured in config/ai.yml: - 1. claude -p (Claude CLI with subagent) - 2. codex exec --model gpt-5 --json - 3. gemini --model gemini-2.5-flash - - Each provider tried until one succeeds. - Provides redundancy against: - - Rate limits - - API outages - - Non-diff responses -end note - -note right of patcher - Saves debug artifacts to - .git/ai-rules-debug/ - for troubleshooting: - - *.raw.out - - *.clean.diff - - *.sanitized.diff - - *.final.diff -end note - -note right of workflow - Always exits 0 - (non-blocking) - - Extracts structured markers: - - **DECISION**: text - - **QUESTION**: text - - **ACTION**: @assignee text -end note - -@enduml diff --git a/docs/diagrams-README.md b/docs/diagrams-README.md deleted file mode 100644 index ebcb899..0000000 --- a/docs/diagrams-README.md +++ /dev/null @@ -1,275 +0,0 @@ -# CascadingDev Architecture Diagrams - -This directory contains PlantUML diagrams documenting the CascadingDev automation system. - -## Viewing the Diagrams - -### Option 1: VS Code (Recommended) -1. Install the [PlantUML extension](https://marketplace.visualstudio.com/items?itemName=jebbs.plantuml) -2. Open any `.puml` file -3. Press `Alt+D` to preview - -### Option 2: Online Viewer -Visit [PlantUML Web Server](http://www.plantuml.com/plantuml/uml/) and paste the diagram content. - -### Option 3: Command Line -```bash -# Install PlantUML -sudo apt install plantuml # or brew install plantuml - -# Generate PNG -plantuml docs/architecture-overview.puml - -# Generate SVG (better quality) -plantuml -tsvg docs/*.puml -``` - -## Diagram Index - -### 1. **architecture-overview.puml** -**High-level system architecture** showing how all components interact. - -**Shows:** -- User project structure -- Automation modules (runner, config, patcher, workflow) -- AI provider integration -- Data flow from developer to commit - -**Best for:** Understanding the big picture and component relationships. - ---- - -### 2. **commit-workflow.puml** ⭐ UPDATED -**Sequence diagram** of what happens during `git commit` with multi-provider fallback. - -**Shows:** -- Pre-commit hook execution -- runner.py orchestration -- Multi-provider AI fallback (claude → codex → gemini) -- Patch generation and application -- Vote processing with marker extraction -- File staging - -**Best for:** Understanding the complete commit-time automation flow with provider redundancy. - ---- - -### 3. **cascading-rules.puml** -**Configuration system** showing how `.ai-rules.yml` files cascade and merge. - -**Shows:** -- Rule file locations (root, features/, FR_*/ -- Merge precedence (nearest wins) -- File associations -- Rule definitions - -**Best for:** Understanding how to configure automation rules. - ---- - -### 4. **patcher-pipeline.puml** ⭐ UPDATED -**Detailed flowchart** of AI patch generation and application with provider fallback. - -**Shows:** -- Prompt building with model hints -- Multi-provider fallback logic (claude → codex → gemini) -- Codex JSON parsing -- Patch extraction (with markers) -- Patch sanitization -- git apply strategies (strict → 3-way → fallback) -- Debug artifact saving -- Error handling - -**Best for:** Debugging patch application issues or understanding the AI provider chain. - ---- - -### 5. **voting-system.puml** ⭐ UPDATED -**Voting and promotion logic** for multi-stage feature discussions. - -**Shows:** -- Vote parsing from discussion files -- Eligible voter calculation -- Stage-specific promotion thresholds (READY_FOR_DESIGN, READY_FOR_IMPLEMENTATION, etc.) -- Rejection logic per stage -- Summary file updates - -**Best for:** Understanding how features move through multi-stage approval (feature → design → implementation → review). - ---- - -### 6. **file-lifecycle.puml** -**Activity diagram** showing the complete lifecycle of a feature request. - -**Shows:** -- From `request.md` creation to implementation -- AI-generated file creation -- Vote-driven status transitions -- Implementation gate triggering -- Which files to edit vs. auto-generated - -**Best for:** Understanding the developer workflow and when automation triggers. - ---- - -### 7. **directory-structure.puml** -**Component diagram** showing the project structure. - -**Shows:** -- CascadingDev repository layout -- Install bundle structure -- User project structure after setup -- Debug artifact locations -- File relationships - -**Best for:** Navigating the codebase and understanding where files live. - ---- - -### 8. **ai-provider-fallback.puml** 🆕 -**Detailed flowchart** of the multi-provider AI fallback chain with model hints. - -**Shows:** -- Command chain selection (default, fast, quality) -- Model hint propagation (TASK COMPLEXITY) -- Provider-specific execution (Claude CLI, Codex JSON, Gemini) -- Fallback logic (claude → codex → gemini) -- Sentinel token handling -- Error handling per provider - -**Best for:** Understanding how AI provider redundancy works and debugging provider failures. - ---- - -### 9. **discussion-stages.puml** 🆕 -**State machine diagram** showing the complete feature discussion lifecycle through all stages. - -**Shows:** -- Feature stage (OPEN → READY_FOR_DESIGN) -- Design stage (OPEN → READY_FOR_IMPLEMENTATION) -- Implementation stage (IN_PROGRESS → READY_FOR_REVIEW) -- Review stage (UNDER_REVIEW → APPROVED) -- Status transitions and promotion conditions -- Auto-generated files per stage - -**Best for:** Understanding the full feature approval workflow from request to merge. - ---- - -### 10. **workflow-marker-extraction.puml** 🆕 -**Detailed flowchart** of AI-powered marker extraction with simple fallback parsing. - -**Shows:** -- Comment parsing from discussion files -- AI normalization (agents.py) for natural conversation -- Simple line-start fallback for explicit markers (DECISION:, QUESTION:, ACTION:) -- Structured data extraction from AI-generated JSON -- Summary section generation -- Marker block updates in .sum.md files - -**Best for:** Understanding the two-tier extraction system - AI for natural conversation, simple parsing for strict format fallback. - ---- - -## Key Concepts Illustrated - -### Automation Pipeline -``` -Developer commits → Pre-commit hook → runner.py → patcher.py → Multi-Provider AI - → config.py → .ai-rules.yml (claude → codex → gemini) - → workflow.py → summary.py - (regex marker extraction) -``` - -### File Processing (Multi-Stage) -``` -request.md (edited) → AI generates → feature.discussion.md (created/updated) - → feature.discussion.sum.md (updated) - ↓ (votes → READY_FOR_DESIGN) - → design.discussion.md (created) - → design.discussion.sum.md (updated) - ↓ (votes → READY_FOR_IMPLEMENTATION) - → implementation.discussion.md (created) - ↓ (votes → READY_FOR_REVIEW) - → review.discussion.md (created) -``` - -### Voting Flow -``` -Participant comments → VOTE: lines parsed → Count eligible votes - → Check thresholds - → Update status - → Generate implementation (if ready) -``` - -### Error Handling -``` -API overload → Log error → Continue with other files → Commit succeeds -Patch fails → Save debug artifacts → Log error → Continue -``` - ---- - -## Common Workflows - -### Adding a New Rule -1. See **cascading-rules.puml** for rule structure -2. Edit `Docs/features/.ai-rules.yml` -3. Add `file_associations` and `rules` sections -4. Commit and test - -### Debugging Automation -1. Check `.git/ai-rules-debug/*.raw.out` for AI responses -2. See **patcher-pipeline.puml** for patch processing steps -3. Review **commit-workflow.puml** for execution order - -### Understanding Status Transitions -1. See **discussion-stages.puml** for complete multi-stage flow -2. See **voting-system.puml** for promotion logic per stage -3. Check **file-lifecycle.puml** for developer workflow -4. Review discussion file YAML headers for thresholds - -### Understanding AI Provider System -1. See **ai-provider-fallback.puml** for provider chain logic -2. Check **patcher-pipeline.puml** for integration details -3. Review config/ai.yml for command configuration -4. Check .git/ai-rules-debug/ for provider outputs - -### Understanding Marker Extraction -1. See **workflow-marker-extraction.puml** for AI normalization flow -2. Review automation/agents.py for AI-powered extraction -3. Review automation/workflow.py for simple fallback implementation -4. Test with natural conversation - AI extracts markers automatically -5. Fallback: Use explicit line-start markers (DECISION:, QUESTION:, ACTION:) - ---- - -## Implementation Status - -| Feature | Status | Diagram | -|---------|--------|---------| -| Cascading Rules | ✅ Complete | cascading-rules.puml | -| AI Patch Generation | ✅ Complete | patcher-pipeline.puml | -| Multi-Provider Fallback | ✅ Complete | ai-provider-fallback.puml | -| Model Hints (fast/quality) | ✅ Complete | ai-provider-fallback.puml | -| Vote Tracking | ✅ Complete | voting-system.puml | -| Multi-Stage Promotion | ✅ Complete | discussion-stages.puml | -| AI Marker Normalization | ✅ Complete | workflow-marker-extraction.puml | -| Structured Summaries | ✅ Complete | workflow-marker-extraction.puml | -| Implementation Gate | ✅ Complete | file-lifecycle.puml | -| Error Handling | ✅ Complete | commit-workflow.puml | - ---- - -## Related Documentation - -- **[DESIGN.md](DESIGN.md)** - Complete system design document -- **[AUTOMATION.md](AUTOMATION.md)** - Automation system details -- **[automation/README.md](../automation/README.md)** - Quick reference guide -- **[CLAUDE.md](../CLAUDE.md)** - AI assistant guide for this repo - ---- - -**Created:** 2025-10-31 -**Last Updated:** 2025-10-31 -**Diagrams:** 7 total (PlantUML format) diff --git a/docs/directory-structure.puml b/docs/directory-structure.puml deleted file mode 100644 index d907898..0000000 --- a/docs/directory-structure.puml +++ /dev/null @@ -1,123 +0,0 @@ -@startuml directory-structure -!theme plain -title CascadingDev Project Directory Structure - -folder "CascadingDev Repository" { - folder "automation/" as repo_auto #LightBlue { - file "runner.py" as runner #SkyBlue - file "config.py" as config #SkyBlue - file "patcher.py" as patcher #SkyBlue - file "workflow.py" as workflow #SkyBlue - file "agents.py" as agents #SkyBlue - file "summary.py" as summary #SkyBlue - file "README.md" as auto_readme - } - - folder "assets/" #LightGreen { - folder "hooks/" { - file "pre-commit" #LightCoral - } - folder "templates/" { - file "feature_request.md" - file "feature.discussion.md" - file "feature.discussion.sum.md" - folder "rules/" { - file "root.ai-rules.yml" - file "features.ai-rules.yml" - } - } - } - - folder "tests/" #LightYellow { - file "test_workflow.py" - file "test_config.py" - file "test_patcher.py" - file "test_runner.py" - } - - folder "tools/" { - file "build_installer.py" - file "mock_ai.sh" - } - - folder "docs/" #Lavender { - file "DESIGN.md" - file "AUTOMATION.md" - file "architecture-overview.puml" #Pink - file "commit-workflow.puml" #Pink - file "cascading-rules.puml" #Pink - file "patcher-pipeline.puml" #Pink - file "voting-system.puml" #Pink - file "file-lifecycle.puml" #Pink - } - - file ".ai-rules.yml" #Orange - file "pyproject.toml" -} - -folder "Install Bundle\n(Built by build_installer.py)" #LightGray { - folder "automation/" as install_auto #LightBlue - folder "assets/" as install_assets #LightGreen - folder "process/templates/" #Wheat - file "setup_cascadingdev.py" #Coral -} - -folder "User Project\n(After setup)" #Wheat { - folder "Docs/features/" { - file ".ai-rules.yml" #Orange - folder "FR_2025-10-31_feature-name/" { - file "request.md" #LightGreen - folder "discussions/" { - file "feature.discussion.md" #SkyBlue - file "feature.discussion.sum.md" #LightBlue - file "implementation.discussion.md" #SkyBlue - file "implementation.discussion.sum.md" #LightBlue - } - } - } - - folder "automation/" as user_auto #LightBlue { - note as auto_note - Copied from install bundle - Runs during git commits - end note - } - - folder ".git/hooks/" { - file "pre-commit" #LightCoral - } - - folder ".git/ai-rules-debug/" #Pink { - file "*.raw.out" - file "*.clean.diff" - file "*.sanitized.diff" - file "*.final.diff" - - note as debug_note - Debug artifacts saved here - when automation runs - end note - } -} - -note top of runner - **Entrypoint for AI automation** - Called by pre-commit hook - Processes staged files - according to .ai-rules.yml -end note - -note top of patcher - **AI patch generation** - Calls Claude API - Applies patches with git -end note - -note top of workflow - **Vote tracking & summaries** - Parses VOTE: lines - Updates .sum.md files - Always runs (no AI needed) -end note - -@enduml diff --git a/docs/discussion-processing.puml b/docs/discussion-processing.puml deleted file mode 100644 index 29b2b8e..0000000 --- a/docs/discussion-processing.puml +++ /dev/null @@ -1,85 +0,0 @@ -@startuml discussion-processing -!theme plain -title Discussion Automation Pipeline (current behaviour) - -start - -:Developer stages discussion file; -:Pre-commit hook invokes\nautomation/workflow.py:_run_status(); - -partition "Vote Handling" { - :Parse staged discussion snapshot\n(parse_votes); - :Print vote summary to console; - if (Promotion rule present?) then (yes) - :count_eligible_votes(); - :check_promotion_threshold(); - if (Threshold met?) then (yes) - :update_discussion_status(); - :git add updated discussion; - endif - endif -} - -partition "Incremental Extraction" { - :get_discussion_changes()\n(staged diff → added lines only); - :Call extract_structured_basic()\n(simple marker parsing); - partition "AI Normalizer" { - :process_discussion_with_ai(); - if (Providers available?) then (yes) - repeat - :Call normalize_discussion()\n(claude → codex → gemini); - if (Diff/JSON valid?) then (yes) - stop - else (no / malformed) - :Raise PatchGenerationError; - :Fallback to next provider; - endif - repeat while (more providers) - endif - if (AI returned data?) then (yes) - :Merge AI JSON onto structured result; - else (no) - :Keep regex-only extraction; - endif - } -} - -partition "Summary Update" { - :Load companion summary file (if any); - :load_summary_state()\n(read <!-- SUMMARY:STATE --> JSON); - :merge_* helpers update\n questions / actions / decisions / mentions; - :save_summary_state(); - :format_votes_section(); - :format_questions_section(); - :format_action_items_section(); - :format_decisions_section(); - :format_awaiting_section(); - :append_timeline_entry(); - :Write summary file and git add; -} - -partition "Design / Implementation Outputs" { - :RulesConfig routes staged file; - if (feature discussion promoted to READY_FOR_DESIGN?) then (yes) - :design_gate_writer ensures design discussion exists; - endif - if (design discussion promoted to READY_FOR_IMPLEMENTATION?) then (yes) - :implementation_gate_writer creates\nimplementation discussion; - endif - if (design discussion updated?) then (yes) - :design_discussion_update rule triggers; - :design_discussion_writer appends AI comment; - :design_doc_writer (new rule)\nupdates Docs/.../design/design.md\nvia provider chain; - endif -} - -stop - -legend bottom - **Key Notes** - - AI providers are attempted in order (claude → codex → gemini); malformed diffs raise PatchGenerationError so the next provider runs. - - Summary state persists in <!-- SUMMARY:STATE {...} --> and is rewritten every commit. - - Vote thresholds drive status changes which gate creation of downstream discussions/docs. -endlegend - -@enduml diff --git a/docs/discussion-stages.puml b/docs/discussion-stages.puml deleted file mode 100644 index 7c616e0..0000000 --- a/docs/discussion-stages.puml +++ /dev/null @@ -1,153 +0,0 @@ -@startuml discussion-stages -!theme plain -title Feature Discussion Stage Progression with Status Transitions - -state "Feature Stage" as feature { - [*] --> OPEN_F : feature.discussion.md created - - OPEN_F : status: OPEN - OPEN_F : Participants discuss scope,\nplatform, requirements - - OPEN_F --> READY_FOR_DESIGN : ≥2 READY votes\n(human only if\nallow_agent_votes: false) - - READY_FOR_DESIGN : status: READY_FOR_DESIGN - READY_FOR_DESIGN : Scope approved,\nready for technical design - - READY_FOR_DESIGN --> FEATURE_REJECTED : Majority REJECT votes - FEATURE_REJECTED : status: FEATURE_REJECTED - FEATURE_REJECTED : Feature blocked - - note right of READY_FOR_DESIGN - **AI Auto-generates:** - design.discussion.md - (Initial design proposal) - end note -} - -state "Design Stage" as design { - [*] --> OPEN_D : design.discussion.md created - - OPEN_D : status: OPEN - OPEN_D : Discuss architecture,\ntech stack, data models - - OPEN_D --> READY_FOR_IMPLEMENTATION : ≥2 READY votes - READY_FOR_IMPLEMENTATION : status: READY_FOR_IMPLEMENTATION - READY_FOR_IMPLEMENTATION : Design approved,\nready to implement - - OPEN_D --> DESIGN_REJECTED : Majority REJECT votes - DESIGN_REJECTED : status: DESIGN_REJECTED - DESIGN_REJECTED : Design needs rework - - note right of READY_FOR_IMPLEMENTATION - **AI Auto-generates:** - implementation.discussion.md - (Implementation tracking) - end note -} - -state "Implementation Stage" as impl { - [*] --> OPEN_I : implementation.discussion.md created - - OPEN_I : status: OPEN - OPEN_I : Track engineering tasks,\nprogress, and blockers - - OPEN_I --> READY_FOR_TESTING : All checkboxes complete\nAND ≥1 human READY vote - - READY_FOR_TESTING : status: READY_FOR_TESTING - READY_FOR_TESTING : Implementation complete - - note right of READY_FOR_TESTING - **Automation syncs:** - implementation/tasks.md - (checkbox mirror + summary) - end note -} - -state "Review Stage" as review { - [*] --> UNDER_REVIEW : review.discussion.md created - - UNDER_REVIEW : status: UNDER_REVIEW - UNDER_REVIEW : Code review,\ntesting, QA - - UNDER_REVIEW --> APPROVED : ≥2 READY votes - UNDER_REVIEW --> NEEDS_CHANGES : CHANGES votes - - APPROVED : status: APPROVED - APPROVED : Ready to merge - - NEEDS_CHANGES --> UNDER_REVIEW : Changes addressed -} - -[*] --> feature -READY_FOR_DESIGN --> design -READY_FOR_IMPLEMENTATION --> impl -READY_FOR_TESTING --> review -APPROVED --> [*] - -legend bottom - **Vote Counting Rules (Configured per stage):** - - promotion_rule: - allow_agent_votes: false # Only count human votes - ready_min_eligible_votes: 2 # Need 2 READY to promote - reject_min_eligible_votes: 1 # Need 1 REJECT to block - - **Vote Format in Discussion Files:** - Name: ParticipantName - Comment text. - VOTE: READY|CHANGES|REJECT - - Name: AI_BotName - Comment text. - VOTE: CHANGES (excluded if allow_agent_votes: false) - - **Status Transitions:** - - Automatic when vote thresholds met - - AI updates YAML header: status: field - - AI appends consensus comment - - Triggers next stage discussion creation - - **Example Progression:** - 1. Feature discussion: Define WHAT we're building - 2. Design discussion: Define HOW we'll build it - 3. Implementation discussion: Track building progress - 4. Review discussion: Verify quality before merge -endlegend - -note right of feature - **Participants Use Structured Markers:** - **DECISION**: Web platform, React frontend - **QUESTION**: Mobile support in MVP? - **ACTION**: @Alice research auth options - - These are extracted to .sum.md files - for easy reference -end note - -note right of design - **Design Decisions Tracked:** - - Architecture choices - - Technology stack - - Data models - - API contracts - - Risk trade-offs -end note - -note right of impl - **Implementation Tracking:** - - Task breakdowns - - Blocking issues - - Progress updates - - Code references -end note - -note right of review - **Review Checklist:** - - Code quality - - Test coverage - - Documentation - - Performance - - Security -end note - -@enduml diff --git a/docs/file-lifecycle.puml b/docs/file-lifecycle.puml deleted file mode 100644 index 073813d..0000000 --- a/docs/file-lifecycle.puml +++ /dev/null @@ -1,120 +0,0 @@ -@startuml file-lifecycle -!theme plain -title Feature File Lifecycle and Automation Triggers - -|Developer| -start -:Create feature request; -:Edit **request.md**; - -|Git| -:git add request.md; -:git commit; - -|Pre-commit Hook| -:runner.py processes\nrequest.md; - -|AI (runner + patcher)| -if (feature.discussion.md exists?) then (no) - :Generate new\n**feature.discussion.md**: - - YAML header with promotion rules - - Summary from request - - Initial AI comment + vote; -else (yes) - :Append AI comment to\nexisting discussion; -endif - -:Stage feature.discussion.md; - -if (feature.discussion.sum.md exists?) then (no) - :Create from template; -else (yes) - :Update summary sections; -endif - -:Stage feature.discussion.sum.md; - -|Git| -:Commit completes with\nauto-generated files; - -|Developer| -:Review generated discussion; - -if (Make changes?) then (yes) - :Edit **feature.discussion.md**; - :Add your comment + VOTE; - - |Git| - :git add feature.discussion.md; - :git commit; - - |Pre-commit Hook| - :workflow.py parses votes; - :runner.py appends AI response; - - |AI| - :Read all comments + votes; - :Calculate promotion status; - - if (READY votes >= threshold?) then (yes) - :Update status to\n**READY_FOR_IMPLEMENTATION**; - - if (implementation_gate enabled?) then (yes) - :Generate\n**implementation.discussion.md**; - :Stage implementation file; - endif - endif - - :Append new AI comment\nwith vote analysis; - :Stage updated discussion; - - |Git| - :Commit with updates; -endif - -|Developer| -if (Status == READY_FOR_IMPLEMENTATION?) then (yes) - :Begin implementation; - :Edit **implementation.discussion.md**; - :Track tasks and progress; - - |Pre-commit Hook| - :AI adds planning updates; - :Updates task checklists; - - |Git| - :Commit implementation progress; -endif - -stop - -note right - **Files Auto-Generated:** - 1. feature.discussion.md - 2. feature.discussion.sum.md - 3. implementation.discussion.md (gated) - - **Never Edit These Manually:** - - .sum.md files (always auto-generated) - - **Edit These Freely:** - - request.md (your feature spec) - - *.discussion.md (add your comments) -end note - -note right - **Two Automation Phases:** - - **Phase 1 - Vote Tracking:** - - Always runs (no AI needed) - - Parses VOTE: lines - - Updates .sum.md sections - - **Phase 2 - AI Enhancement:** - - Requires Claude API/CLI - - Generates intelligent comments - - Tracks quorum and status - - Creates gated files -end note - -@enduml diff --git a/docs/patcher-pipeline.puml b/docs/patcher-pipeline.puml deleted file mode 100644 index 633bd9a..0000000 --- a/docs/patcher-pipeline.puml +++ /dev/null @@ -1,149 +0,0 @@ -@startuml patcher-pipeline -!theme plain -title AI Patch Generation and Application Pipeline - -start - -:Receive source file + target file + instruction; - -:Build prompt with: -- Source file diff (staged changes) -- Source file full content -- Target file current content -- Generation instructions from rules -- Model hint (fast/quality) if specified; - -partition "Multi-Provider Fallback" { - :Try Provider 1: Claude CLI; - - if (Claude returned output?) then (yes) - if (Output contains\n"API Error: Overloaded"?) then (yes) - :Raise PatchGenerationError\n"Claude API is overloaded"; - stop - endif - - if (Output contains diff markers?) then (yes) - :Success! Continue to extraction; - else (no - non-diff response) - :Log: "Claude non-diff output"; - :Try Provider 2: Codex CLI; - - if (Codex returned output?) then (yes) - :Parse JSON response\nextract agent_message; - - if (Parsed text contains diff?) then (yes) - :Success! Continue to extraction; - else (no - exit code 1) - :Log: "Codex exited with 1"; - :Try Provider 3: Gemini CLI; - - if (Gemini returned output?) then (yes) - if (Gemini returned sentinel?) then (yes) - :Log: "No changes needed"; - stop - else (has diff) - :Success! Continue to extraction; - endif - else (no) - :Raise "All providers failed"; - stop - endif - endif - else (no) - :Raise "All providers failed"; - stop - endif - endif - else (no - command failed) - :Raise "Provider 1 command failed"; - stop - endif -} - -:Save raw output to\n.git/ai-rules-debug/*.raw.out; - -if (Output contains\n<<<AI_DIFF_START>>>?) then (yes) - :Extract content between\nSTART and END markers; -else (no) - if (Output contains\n"diff --git"?) then (yes) - :Extract from\n"diff --git" onward; - else (no) - :Raise "AI output did not contain a diff"; - stop - endif -endif - -:Save to *.clean.diff; - -:Sanitize patch: -- Remove "index ..." lines -- Remove "similarity index" lines -- Keep only diff content; - -:Save to *.sanitized.diff; - -if (New file and missing\n"new file mode"?) then (yes) - :Add "new file mode 100644" header; -endif - -:Save to *.final.diff; - -if (Patch is empty?) then (yes) - :Raise "AI returned empty patch"; - stop -endif - -:Try git apply -p1 --index --check; - -if (Check succeeded?) then (yes) - :git apply -p1 --index; - :Success!; - stop -endif - -:Try git apply -p1 --index --3way\n--recount --whitespace=nowarn; - -if (3-way succeeded?) then (yes) - :Applied with 3-way merge; - :Success!; - stop -endif - -if (Is new file?) then (yes) - :Try git apply -p1 (without --index); - if (Succeeded?) then (yes) - :git add target file; - :Success!; - stop - endif -endif - -:Raise "Failed to apply patch\n(strict and 3-way both failed)"; -stop - -note right - **AI Provider Configuration:** - config/ai.yml defines fallback chains: - - command_chain (default) - - command_chain_fast (model_hint: fast) - - command_chain_quality (model_hint: quality) - - **Provider Details:** - 1. Claude: claude -p (auto-selects subagent) - 2. Codex: codex exec --model gpt-5 --json - 3. Gemini: gemini --model gemini-2.5-flash - - **Debug Artifacts Location:** - .git/ai-rules-debug/ - - Files saved: - - *.raw.out (full AI response) - - *.clean.diff (extracted patch) - - *.sanitized.diff (cleaned patch) - - *.final.diff (applied patch) - - Filename format: - {output_path_with_underscores}-{pid}.{ext} -end note - -@enduml diff --git a/docs/voting-system.puml b/docs/voting-system.puml deleted file mode 100644 index 29d56a2..0000000 --- a/docs/voting-system.puml +++ /dev/null @@ -1,124 +0,0 @@ -@startuml voting-system -!theme plain -title Feature Discussion Voting and Promotion System - -start - -:Discussion file updated; - -partition "Vote Parsing (workflow.py)" { - :Read staged discussion.md content; - - :Parse all lines matching: - **- ParticipantName: ... VOTE: VALUE**; - - :Track latest vote per participant\n(most recent wins); - - :Count eligible voters based on\n**allow_agent_votes** rule; - - note right - **Vote Format:** - Name: ParticipantName - Comment text. - VOTE: READY - - Name: AI_BotName - Comment. - VOTE: CHANGES - - **Valid Values:** - - READY (approve) - - CHANGES (needs work) - - REJECT (block) - end note -} - -partition "Promotion Logic (AI-powered)" { - :AI reads promotion_rule from header: - - allow_agent_votes: true/false - - ready_min_eligible_votes: N or "all" - - reject_min_eligible_votes: N or "all"; - - if (allow_agent_votes == false?) then (yes) - :Exclude voters with\nnames starting with "AI_"; - endif - - :Count eligible READY votes; - :Count eligible REJECT votes; - :Count CHANGES votes (neutral); - - if (READY threshold met AND\nREJECT threshold NOT met?) then (yes) - if (Current stage == feature?) then (yes) - :Update status to\n**READY_FOR_DESIGN**; - :AI generates\ndesign.discussion.md; - else if (Current stage == design?) then (yes) - :Update status to\n**READY_FOR_IMPLEMENTATION**; - :AI generates\nimplementation.discussion.md; - else if (Current stage == implementation?) then (yes) - :Verify all checkboxes checked\nAND ≥1 human READY vote; - if (Requirements met?) then (yes) - :Update status to\n**READY_FOR_TESTING**; - :AI generates\ntesting discussion artefacts; - else - :Keep status **OPEN** (await more progress); - endif - else (review stage) - :Update status to\n**APPROVED**; - endif - else if (REJECT threshold met AND\nREADY threshold NOT met?) then (no) - if (Current stage == feature?) then (yes) - :Update status to\n**FEATURE_REJECTED**; - else if (Current stage == design?) then (yes) - :Update status to\n**DESIGN_REJECTED**; - else (other stages) - :Update status to\n**NEEDS_CHANGES**; - endif - else (no) - :Keep status as **OPEN** or **UNDER_REVIEW**; - endif -} - -partition "Summary Update (summary.py)" { - :Update VOTES section in .sum.md; - note right - Example block written into the summary file: - <!-- SUMMARY:VOTES START --> - ## Votes (latest per participant) - READY: X • CHANGES: Y • REJECT: Z - - Alice: READY - - Bob: CHANGES - <!-- SUMMARY:VOTES END --> - end note - - :Auto-stage updated .sum.md file; -} - -:Include in commit; - -stop - -legend bottom - Example Promotion Rules: - - Simple Majority (2 approvals): - ready_min_eligible_votes: 2 - reject_min_eligible_votes: 1 - allow_agent_votes: false - - Unanimous (everyone must approve): - ready_min_eligible_votes: "all" - reject_min_eligible_votes: 1 - allow_agent_votes: false - - Include AI votes: - ready_min_eligible_votes: 3 - allow_agent_votes: true - - Implementation human gate: - ready_min_eligible_votes: 1 - allow_agent_votes: true - # workflow enforces ≥1 human READY - # and completion of all tasks -endlegend - -@enduml diff --git a/docs/workflow-marker-extraction.puml b/docs/workflow-marker-extraction.puml deleted file mode 100644 index f635635..0000000 --- a/docs/workflow-marker-extraction.puml +++ /dev/null @@ -1,141 +0,0 @@ -@startuml workflow-marker-extraction -!theme plain -title Workflow Marker Extraction with AI Normalization - -start - -:Discussion file staged\n(feature.discussion.md,\ndesign.discussion.md, etc); - -:workflow.py reads file content; - -partition "Two-Tier Extraction" { - :Call extract_structured_basic()\nSimple fallback parsing; - - note right - **Fallback: Simple Line-Start Matching** - Only matches explicit markers at line start: - - DECISION: text - - QUESTION: text - - Q: text - - ACTION: text - - TODO: text - - ASSIGNED: text - - DONE: text - - Uses case-insensitive startswith() matching. - Handles strictly-formatted discussions. - end note - - :Store fallback results\n(decisions, questions, actions, mentions); - - :Call agents.normalize_discussion()\nAI-powered extraction; - - partition "AI Normalization (agents.py)" { - :Build prompt for AI model; - note right - **AI Prompt:** - "Extract structured information from discussion. - Return JSON with: votes, questions, decisions, - action_items, mentions" - - Supports natural conversation like: - "I'm making a decision here - we'll use X" - "Does anyone know if we need Y?" - "@Sarah can you check Z?" - end note - - :Execute command chain\n(claude → codex → gemini); - - if (AI returned valid JSON?) then (yes) - :Parse JSON response; - :Extract structured data:\n- votes\n- questions\n- decisions\n- action_items\n- mentions; - :Override fallback results\nwith AI results; - note right - **AI advantages:** - - Handles embedded markers - - Understands context - - Extracts from natural language - - No strict formatting required - end note - else (no - AI failed or unavailable) - :Use fallback results only; - note right - **Fallback activated when:** - - All providers fail - - Invalid JSON response - - agents.py import fails - - API rate limits hit - end note - endif - } -} - -partition "Generate Summary Sections" { - :Format Decisions section:\n- Group by participant\n- Number sequentially\n- Include rationale if present; - - :Format Open Questions section:\n- List unanswered questions\n- Track by participant\n- Mark status (OPEN/PARTIAL); - - :Format Action Items section:\n- Group by status (TODO/ASSIGNED/DONE)\n- Show assignees\n- Link to requesters; - - :Format Awaiting Replies section:\n- Group by @mentioned person\n- Show context of request\n- Track unresolved mentions; - - :Format Votes section:\n- Count by value (READY/CHANGES/REJECT)\n- List latest vote per participant\n- Exclude AI votes if configured; - - :Format Timeline section:\n- Chronological order (newest first)\n- Include status changes\n- Summarize key events; -} - -:Update marker blocks in .sum.md; -note right - <!-- SUMMARY:DECISIONS START --> - ... - <!-- SUMMARY:DECISIONS END --> -end note - -:Stage updated .sum.md file; - -stop - -legend bottom - **Example Input (natural conversation):** - - Name: Rob - I've been thinking about the timeline. I'm making a decision here - - we'll build the upload system first. Does anyone know if we need real-time - preview? @Sarah can you research Unity Asset Store API? - VOTE: READY - - **AI Normalization Output (JSON):** - { - "votes": [{"participant": "Rob", "vote": "READY"}], - "decisions": [{"participant": "Rob", - "decision": "build the upload system first"}], - "questions": [{"participant": "Rob", - "question": "if we need real-time preview"}], - "action_items": [{"participant": "Rob", "action": "research Unity API", - "assignee": "Sarah"}], - "mentions": [{"from": "Rob", "to": "Sarah"}] - } - - **Fallback Only Matches:** - DECISION: We'll build upload first - QUESTION: Do we need real-time preview? - ACTION: @Sarah research Unity API -endlegend - -note right - **Architecture Benefits:** - - ✓ Participants write naturally - ✓ No strict formatting rules - ✓ AI handles understanding - ✓ Simple code for fallback - ✓ Resilient (multi-provider chain) - ✓ Cost-effective (fast models) - - **Files:** - - automation/agents.py (AI normalization) - - automation/workflow.py (fallback + orchestration) - - automation/patcher.py (provider chain execution) -end note - -@enduml diff --git a/docs/workflow-marker-extraction.svg b/docs/workflow-marker-extraction.svg deleted file mode 100644 index e7fd675..0000000 --- a/docs/workflow-marker-extraction.svg +++ /dev/null @@ -1,151 +0,0 @@ -<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="103px" preserveAspectRatio="none" style="width:352px;height:103px;background:#000000;" version="1.1" viewBox="0 0 352 103" width="352px" zoomAndPan="magnify"><defs/><g><rect fill="#11060A" height="1" style="stroke: #11060A; stroke-width: 1.0;" width="1" x="0" y="0"/><rect fill="#33FF02" height="24.0679" style="stroke: #33FF02; stroke-width: 1.0;" width="346" x="5" y="5"/><text fill="#000000" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacingAndGlyphs" textLength="344" x="6" y="20">[From workflow-marker-extraction.puml (line 2) ]</text><text fill="#33FF02" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacingAndGlyphs" textLength="0" x="9" y="43.0679"/><text fill="#33FF02" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacingAndGlyphs" textLength="275" x="5" y="62.1358">@startuml workflow-marker-extraction</text><text fill="#33FF02" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacingAndGlyphs" textLength="87" x="5" y="81.2038">!theme plain</text><text fill="#FF0000" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacingAndGlyphs" textLength="93" x="9" y="100.2717">Syntax Error?</text><!--MD5=[32d7802434cc4c797d2bc79c191390cf] -@startuml workflow-marker-extraction -!theme plain -title Workflow Marker Extraction with AI Normalization - -start - -:Discussion file staged\n(feature.discussion.md,\ndesign.discussion.md, etc); - -:workflow.py reads file content; - -partition "Two-Tier Extraction" { - :Call extract_structured_basic()\nSimple fallback parsing; - - note right - **Fallback: Simple Line-Start Matching** - Only matches explicit markers at line start: - - DECISION: text - - QUESTION: text - - Q: text - - ACTION: text - - TODO: text - - ASSIGNED: text - - DONE: text - - Uses case-insensitive startswith() matching. - Handles strictly-formatted discussions. - end note - - :Store fallback results\n(decisions, questions, actions, mentions); - - :Call agents.normalize_discussion()\nAI-powered extraction; - - partition "AI Normalization (agents.py)" { - :Build prompt for AI model; - note right - **AI Prompt:** - "Extract structured information from discussion. - Return JSON with: votes, questions, decisions, - action_items, mentions" - - Supports natural conversation like: - "I'm making a decision here - we'll use X" - "Does anyone know if we need Y?" - "@Sarah can you check Z?" - end note - - :Execute command chain\n(claude → codex → gemini); - - if (AI returned valid JSON?) then (yes) - :Parse JSON response; - :Extract structured data:\n- votes\n- questions\n- decisions\n- action_items\n- mentions; - :Override fallback results\nwith AI results; - note right - **AI advantages:** - - Handles embedded markers - - Understands context - - Extracts from natural language - - No strict formatting required - end note - else (no - AI failed or unavailable) - :Use fallback results only; - note right - **Fallback activated when:** - - All providers fail - - Invalid JSON response - - agents.py import fails - - API rate limits hit - end note - endif - } -} - -partition "Generate Summary Sections" { - :Format Decisions section:\n- Group by participant\n- Number sequentially\n- Include rationale if present; - - :Format Open Questions section:\n- List unanswered questions\n- Track by participant\n- Mark status (OPEN/PARTIAL); - - :Format Action Items section:\n- Group by status (TODO/ASSIGNED/DONE)\n- Show assignees\n- Link to requesters; - - :Format Awaiting Replies section:\n- Group by @mentioned person\n- Show context of request\n- Track unresolved mentions; - - :Format Votes section:\n- Count by value (READY/CHANGES/REJECT)\n- List latest vote per participant\n- Exclude AI votes if configured; - - :Format Timeline section:\n- Chronological order (newest first)\n- Include status changes\n- Summarize key events; -} - -:Update marker blocks in .sum.md; -note right - <!- - SUMMARY:DECISIONS START - -> - ... - <!- - SUMMARY:DECISIONS END - -> -end note - -:Stage updated .sum.md file; - -stop - -legend bottom - **Example Input (natural conversation):** - - Rob: I've been thinking about the timeline. I'm making a decision here - - we'll build the upload system first. Does anyone know if we need real-time - preview? @Sarah can you research Unity Asset Store API? VOTE: READY - - **AI Normalization Output (JSON):** - { - "votes": [{"participant": "Rob", "vote": "READY"}], - "decisions": [{"participant": "Rob", - "decision": "build the upload system first"}], - "questions": [{"participant": "Rob", - "question": "if we need real-time preview"}], - "action_items": [{"participant": "Rob", "action": "research Unity API", - "assignee": "Sarah"}], - "mentions": [{"from": "Rob", "to": "Sarah"}] - } - - **Fallback Only Matches:** - DECISION: We'll build upload first - QUESTION: Do we need real-time preview? - ACTION: @Sarah research Unity API -endlegend - -note right - **Architecture Benefits:** - - ✓ Participants write naturally - ✓ No strict formatting rules - ✓ AI handles understanding - ✓ Simple code for fallback - ✓ Resilient (multi-provider chain) - ✓ Cost-effective (fast models) - - **Files:** - - automation/agents.py (AI normalization) - - automation/workflow.py (fallback + orchestration) - - automation/patcher.py (provider chain execution) -end note - -@enduml - -PlantUML version 1.2020.02(Sun Mar 01 06:22:07 AST 2020) -(GPL source distribution) -Java Runtime: OpenJDK Runtime Environment -JVM: OpenJDK 64-Bit Server VM -Java Version: 21.0.8+9-Ubuntu-0ubuntu124.04.1 -Operating System: Linux -Default Encoding: UTF-8 -Language: en -Country: CA ---></g></svg> \ No newline at end of file