# Orchestrated Discussions - Design Document ## Overview **Orchestrated Discussions** is a standalone Python library and CLI for conducting structured multi-agent AI conversations. It enables humans to facilitate discussions between multiple AI personas, each with distinct perspectives, expertise, and voting behavior. ### Project Context This project is part of a three-project ecosystem: 1. **SmartTools** (`~/PycharmProjects/SmartTools`) - Foundation layer - AI provider abstraction with fallback chains - YAML-based tool definitions - Multi-step execution pipelines - CLI and TUI interfaces - **Status**: Mostly complete 2. **Orchestrated Discussions** (this project) - Conversation layer - Multi-agent discussion orchestration - Phase-based workflows with voting - Turn management and @mentions - **Status**: In design/early development 3. **CascadingDev** (`~/PycharmProjects/CascadingDev`) - Integration layer - Git-driven automation with cascading rules - Pre-commit hook orchestration - Will be refactored to depend on SmartTools + Orchestrated Discussions - **Status**: Functional but entangled; needs refactoring after dependencies are ready ### Design Goals 1. **Standalone utility** - Works without git, CascadingDev, or any specific workflow 2. **SmartTools integration** - Uses SmartTools for AI provider abstraction 3. **Extensible personas** - Users can create/customize discussion participants 4. **Multiple interfaces** - CLI, TUI, and Python API 5. **Append-only discussions** - Markdown files that grow, never shrink 6. **Portable state** - Discussion files are self-contained and human-readable --- ## Architecture ### Dependency Graph ``` ┌─────────────────────────────────────┐ │ User Application │ │ (CascadingDev, custom scripts) │ └─────────────────┬───────────────────┘ │ ┌─────────────────▼───────────────────┐ │ Orchestrated Discussions │ │ - Discussion state management │ │ - Phase/turn orchestration │ │ - Voting and consensus │ │ - @mention routing │ └─────────────────┬───────────────────┘ │ ┌─────────────────▼───────────────────┐ │ SmartTools │ │ - AI provider abstraction │ │ - Tool execution engine │ │ - Provider fallback chains │ └─────────────────────────────────────┘ ``` ### Module Structure ``` orchestrated-discussions/ ├── src/discussions/ │ ├── __init__.py # Package exports │ ├── cli.py # CLI entry point (discussions command) │ ├── discussion.py # Discussion class and state management │ ├── participant.py # Participant definitions and loading │ ├── markers.py # Marker parsing (Q:, TODO:, VOTE:, etc.) │ ├── turns.py # Turn management and @mention routing │ ├── phases.py # Phase definitions and transitions │ ├── voting.py # Vote counting and consensus logic │ ├── runner.py # Main orchestration engine │ └── ui/ │ ├── __init__.py │ └── tui.py # Optional urwid-based TUI │ ├── config/ │ └── default_participants.yaml # Bundled persona definitions │ ├── examples/ │ ├── feature_discussion.md # Example: feature planning │ ├── code_review.md # Example: code review │ └── architecture_decision.md # Example: ADR discussion │ ├── tests/ │ ├── test_markers.py │ ├── test_voting.py │ ├── test_phases.py │ └── test_runner.py │ ├── docs/ │ └── DESIGN.md # This document │ ├── pyproject.toml # Package configuration └── README.md # User documentation ``` --- ## Core Concepts ### 1. Discussion A **Discussion** is a markdown file containing: - Metadata (title, phase, status) - Context/description - Participant comments with optional votes - Structured markers (questions, decisions, todos) ```markdown # Feature X Implementation ## Context We need to implement feature X that allows users to... ## Requirements - Must support... - Should integrate with... --- Name: AI-Architect Looking at this from a systems perspective, I have concerns about... The proposed approach could lead to tight coupling between... Q: Have we considered using the adapter pattern here? VOTE: CHANGES --- Name: AI-Pragmatist I think we're overcomplicating this. The simplest approach would be... VOTE: READY --- ``` ### 2. Participant A **Participant** is an AI persona with: - **Name**: Display name (e.g., "AI-Architect", "AI-Security") - **Alias**: Short mention name (e.g., "architect", "security") - **Role**: Brief role description - **Personality**: System prompt defining perspective and behavior - **Expertise**: List of expertise areas - **Concerns**: What this participant watches for - **Vote behavior**: Whether this participant votes (voting vs background) Participants are stored as YAML files and can optionally be backed by SmartTools. ```yaml # config/participants/architect.yaml name: AI-Architect alias: architect role: Systems Architect personality: | You are AI-Architect (also known as Chen), a senior systems architect with deep expertise in distributed systems, design patterns, and long-term technical strategy. Your perspective: - Think 2-5 years ahead, not just immediate implementation - Value modularity, separation of concerns, and clean boundaries - Prefer boring, proven technology over cutting-edge experiments - Call out when shortcuts will create architectural debt When responding: - Consider scalability, maintainability, and evolution over time - Identify architectural risks and technical debt implications - Suggest well-established patterns and proven approaches - Balance ideal architecture with practical constraints expertise: - System design - Scalability - Technical debt - Architectural patterns - API design concerns: - "How does this fit the overall architecture?" - "Will this scale?" - "What's the long-term maintenance burden?" - "Are we creating unnecessary coupling?" type: voting # voting | background provider_hint: claude-sonnet # Preferred AI provider ``` ### 3. Markers **Markers** are structured annotations recognized by the system: | Marker | Description | Example | |--------|-------------|---------| | `VOTE:` | Participant's vote | `VOTE: READY` | | `Q:` | Question | `Q: Have we considered caching?` | | `TODO:` | Action item | `TODO: Research rate limiting options` | | `DECISION:` | Recorded decision | `DECISION: We will use PostgreSQL` | | `ASSIGNED:` | Claimed task | `ASSIGNED: @rob will write the spec` | | `DONE:` | Completed task | `DONE: Spec written and reviewed` | | `CONCERN:` | Raised concern | `CONCERN: Security implications unclear` | | `@alias` | Mention participant | `@architect what do you think?` | ### 4. Phases **Phases** define the stages of a discussion. Each phase has: - **ID**: Unique identifier - **Title**: Human-readable name - **Instructions**: What participants should focus on - **Voting mode**: Whether votes are collected in this phase - **Auto-trigger**: Condition for automatic advancement - **Next phase**: What phase follows ```yaml phases: - id: initial_feedback title: Initial Feedback instructions: | Share your initial thoughts on the proposal. Focus on: - Overall feasibility - Major concerns or risks - Questions that need answering voting_mode: false auto_trigger: all_mentioned_responded next: detailed_review - id: detailed_review title: Detailed Review instructions: | Provide detailed feedback from your area of expertise. Raise specific concerns and suggest alternatives. voting_mode: false auto_trigger: null # Manual advancement next: consensus_vote - id: consensus_vote title: Consensus Vote instructions: | Cast your final vote based on the discussion. - READY: Approve to proceed - CHANGES: Needs modifications (specify what) - REJECT: Fundamental issues (explain why) voting_mode: true auto_trigger: null next: null # Terminal phase ``` ### 5. Voting **Voting** determines discussion outcomes: | Vote | Meaning | |------|---------| | `READY` | Approve - no blocking concerns | | `CHANGES` | Conditional approval - needs specified modifications | | `REJECT` | Block - fundamental issues that must be resolved | **Consensus rules** (configurable): - `threshold_ready`: Fraction of READY votes needed (default: 0.67) - `threshold_reject`: Any REJECT blocks by default (threshold: 0.01) - `human_required`: Whether human approval is mandatory ### 6. Turns A **Turn** is one round of participant responses. The orchestrator: 1. Identifies who should respond (mentioned participants or all) 2. Provides context (discussion content, phase instructions) 3. Collects responses (comment + optional vote) 4. Appends responses to discussion file 5. Checks for phase transitions --- ## Integration with SmartTools ### Option A: Direct Import (Preferred) ```python # discussions/runner.py from smarttools.providers import call_provider, load_providers from smarttools.tool import load_tool, run_tool def get_participant_response(participant, context, callout): """Get AI response for a participant.""" # Build prompt from participant personality + context prompt = f""" {participant.personality} ## Current Discussion {context} ## Your Task {callout if callout else "Provide your perspective on the discussion."} Respond with JSON: {{"comment": "your markdown comment", "vote": "READY|CHANGES|REJECT|null"}} """ # Use SmartTools provider result = call_provider(participant.provider_hint, prompt) if result.success: return parse_response(result.text) else: return None ``` ### Option B: Participants as SmartTools Each participant can be a SmartTool, allowing users to customize via SmartTools' TUI: ```yaml # ~/.smarttools/discussion-architect/config.yaml name: discussion-architect description: Systems architect for discussions category: Discussion arguments: - flag: --context variable: context description: Discussion content - flag: --callout variable: callout default: "" description: Specific question or request steps: - type: prompt prompt: | You are AI-Architect, a senior systems architect... ## Discussion Context {context} ## Request {callout} Respond with JSON: {"comment": "...", "vote": "READY|CHANGES|REJECT|null"} provider: claude-sonnet output_var: response output: "{response}" ``` ### Provider Configuration SmartTools manages provider configuration in `~/.smarttools/providers.yaml`: ```yaml providers: - name: claude-sonnet command: "claude -p --model sonnet" description: "Balanced quality/speed" - name: claude-haiku command: "claude -p --model haiku" description: "Fast, good for simple responses" - name: opencode-deepseek command: "$HOME/.opencode/bin/opencode run --model deepseek/deepseek-chat" description: "Best value, cheap and accurate" ``` --- ## CLI Interface ### Commands ```bash # Create a new discussion discussions new "Feature X" --template feature discussions new "API Redesign" --template architecture-decision # List discussions discussions list discussions list --status open # Show discussion status discussions status feature-x.md discussions status feature-x.md --verbose # Run a turn (invoke specific participants) discussions turn feature-x.md @architect @security discussions turn feature-x.md @all # All registered participants # Advance phase manually discussions advance feature-x.md discussions advance feature-x.md --to detailed_review # Add human comment discussions comment feature-x.md "I agree with the architect's concerns." discussions comment feature-x.md --vote READY "Looks good to me." # Participant management discussions participants list discussions participants add architect --from-file persona.yaml discussions participants add custom-reviewer --interactive discussions participants remove old-participant # Interactive TUI discussions ui feature-x.md discussions ui # Opens discussion browser ``` ### Example Session ```bash $ discussions new "Add user authentication" --template feature Created: discussions/add-user-authentication.md $ discussions turn add-user-authentication.md @architect @security @pragmatist Invoking AI-Architect... Invoking AI-Security... Invoking AI-Pragmatist... Discussion updated with 3 new comments. Votes: READY: 1, CHANGES: 2, REJECT: 0 $ discussions status add-user-authentication.md Discussion: Add user authentication Phase: initial_feedback Status: OPEN Participants (3 responded): AI-Architect: CHANGES - concerned about session management AI-Security: CHANGES - needs threat model AI-Pragmatist: READY - MVP approach is fine Open Questions (2): Q: Should we use JWT or session cookies? (@architect) Q: What's our token expiry policy? (@security) $ discussions comment add-user-authentication.md "Let's use JWT with 1hr expiry." Added comment from Human (Rob). $ discussions advance add-user-authentication.md Advanced to phase: detailed_review ``` --- ## Python API ```python from discussions import Discussion, Participant, Runner # Load or create a discussion discussion = Discussion.load("feature-x.md") # or discussion = Discussion.create( title="Feature X", template="feature", context="We need to implement..." ) # Access discussion state print(discussion.title) # "Feature X" print(discussion.phase) # "initial_feedback" print(discussion.status) # "OPEN" print(discussion.participants) # ["architect", "security", "pragmatist"] # Get structured data print(discussion.votes) # {"AI-Architect": "CHANGES", ...} print(discussion.questions) # [{"text": "...", "author": "...", "status": "open"}] print(discussion.decisions) # [{"text": "...", "author": "...", "supporters": [...]}] # Run a turn runner = Runner() responses = runner.run_turn( discussion=discussion, participants=["architect", "security"], callout="Please review the updated proposal." ) # Add human comment discussion.add_comment( author="Rob", text="I agree with the security concerns.", vote="CHANGES" ) # Check consensus if discussion.has_consensus(): print(f"Consensus reached: {discussion.consensus_result}") # Save changes discussion.save() ``` --- ## Discussion File Format ### Header Block ```markdown ``` ### Content Sections ```markdown # Feature X Implementation ## Context [Description of what's being discussed] ## Requirements - [Requirement 1] - [Requirement 2] ## Constraints - [Constraint 1] ``` ### Comment Blocks ```markdown --- Name: AI-Architect [Markdown content - the participant's comment] Can include: - Multiple paragraphs - Code blocks - Lists - Q: questions - TODO: action items - DECISION: decisions VOTE: CHANGES --- ``` ### Phase Markers ```markdown ``` The `VOTE-RESET` marker indicates that votes before this point should not be counted for the current phase. --- ## Configuration ### Project Configuration ```yaml # ~/.config/discussions/config.yaml (or discussions.yaml in project root) default_participants: - architect - security - pragmatist default_template: feature consensus: threshold_ready: 0.67 threshold_reject: 0.01 human_required: true providers: default: claude-sonnet fallback: - opencode-deepseek - claude-haiku output: directory: ./discussions # Where to store discussion files summary_files: true # Generate .sum.md files ``` ### Participant Registry ```yaml # ~/.config/discussions/participants.yaml participants: - name: AI-Architect alias: architect role: Systems Architect personality: | You are a senior systems architect... type: voting provider_hint: claude-sonnet - name: AI-Security alias: security role: Security Specialist personality: | You are a security specialist... type: voting provider_hint: claude-sonnet - name: AI-Researcher alias: researcher role: Research Assistant personality: | You are a research assistant... type: background # Does not vote provider_hint: claude-haiku ``` --- ## Bundled Participants The following participants are bundled with the package: ### Voting Participants | Name | Alias | Role | Perspective | |------|-------|------|-------------| | AI-Moderator | `moderator` | Discussion Facilitator | Keeps discussions productive, summarizes, calls votes | | AI-Architect | `architect` | Systems Architect | Long-term thinking, scalability, patterns | | AI-Security | `security` | Security Specialist | Threat modeling, vulnerabilities, mitigations | | AI-Pragmatist | `pragmatist` | Shipping Pragmatist | MVP mindset, practical tradeoffs | | AI-Perfectionist | `perfectionist` | Quality Champion | Code quality, testing, documentation | | AI-Designer | `designer` | UX Designer | User experience, accessibility | ### Background Participants (Non-voting) | Name | Alias | Role | Capability | |------|-------|------|------------| | AI-Researcher | `researcher` | Research Assistant | Web research, documentation lookup | | AI-Visualizer | `visualizer` | Diagram Generator | PlantUML diagrams | --- ## Templates ### Feature Discussion Template ```markdown # {title} ## Context [Describe the feature and why it's needed] ## Requirements - [ ] [Requirement 1] - [ ] [Requirement 2] ## Open Questions - [Question 1] ## Constraints - [Constraint 1] --- *Discussion begins below. Participants will be @mentioned to provide feedback.* ``` ### Code Review Template ```markdown # Code Review: {title} ## Changes [Summary of changes or link to diff] ## Areas of Focus - [ ] Architecture - [ ] Security - [ ] Performance - [ ] Testing - [ ] Documentation --- *Review comments below.* ``` ### Architecture Decision Record (ADR) Template ```markdown # ADR: {title} ## Status PROPOSED ## Context [Why is this decision needed?] ## Options Considered ### Option A: [Name] - Pros: ... - Cons: ... ### Option B: [Name] - Pros: ... - Cons: ... ## Decision [To be determined through discussion] ## Consequences [To be determined] --- *Discussion begins below.* ``` --- ## Implementation Plan ### Phase 1: Core Library (MVP) 1. **`markers.py`** - Parse VOTE:, Q:, TODO:, etc. 2. **`discussion.py`** - Discussion class with load/save 3. **`participant.py`** - Participant loading from YAML 4. **`voting.py`** - Vote counting and consensus 5. **`runner.py`** - Basic turn execution using SmartTools **Deliverable**: Python API that can load discussions, run turns, and save results. ### Phase 2: CLI 1. **`cli.py`** - Implement commands: new, status, turn, comment, advance 2. **Templates** - Bundled discussion templates 3. **Participants** - Bundled persona definitions **Deliverable**: Fully functional CLI for managing discussions. ### Phase 3: Advanced Features 1. **`phases.py`** - Phase definitions and auto-transitions 2. **`turns.py`** - @mention routing and turn tracking 3. **Summary generation** - Auto-generate .sum.md files **Deliverable**: Full orchestration with phases and automatic transitions. ### Phase 4: TUI 1. **`ui/tui.py`** - urwid-based interface 2. **Discussion browser** - List and select discussions 3. **Participant manager** - Add/edit personas 4. **Live discussion view** - Watch discussion in real-time **Deliverable**: Interactive TUI for human participation. --- ## Testing Strategy ### Unit Tests - `test_markers.py` - Marker parsing edge cases - `test_voting.py` - Consensus calculation - `test_discussion.py` - State management - `test_phases.py` - Phase transitions ### Integration Tests - `test_runner.py` - Full turn execution (with mock provider) - `test_cli.py` - CLI command integration ### Example ```python # tests/test_voting.py def test_consensus_reached_with_threshold(): votes = { "AI-Architect": "READY", "AI-Security": "READY", "AI-Pragmatist": "CHANGES", } config = VotingConfig(threshold_ready=0.67) result = calculate_consensus(votes, config) assert result.reached == True assert result.outcome == "READY" def test_reject_blocks_consensus(): votes = { "AI-Architect": "READY", "AI-Security": "REJECT", "AI-Pragmatist": "READY", } config = VotingConfig(threshold_reject=0.01) result = calculate_consensus(votes, config) assert result.reached == False assert result.blocked_by == ["AI-Security"] ``` --- ## Migration from CascadingDev Once this project is stable, CascadingDev can be refactored to use it: 1. **Remove** from CascadingDev: - `automation/schema.py` - `automation/workflow.py` (discussion parts) - `automation/orchestrator.py` - `automation/agent_fetcher.py` - `agents/*.py` (voting agents) - `config/participants.yml` 2. **Add** to CascadingDev: - Dependency on `orchestrated-discussions` - Glue code to trigger discussions on commit - Integration with `.ai-rules.yml` for discussion automation 3. **Keep** in CascadingDev: - `automation/runner.py` (cascading rules) - `automation/config.py` (rules resolution) - `automation/patcher.py` (diff generation) - Git hook orchestration --- ## Open Questions 1. **SmartTools import vs subprocess** - Should we import SmartTools directly or call it as a subprocess? Direct import is cleaner but creates tighter coupling. 2. **Participant storage** - Store in `~/.config/discussions/` or `~/.smarttools/`? The latter allows SmartTools TUI editing. 3. **Discussion storage** - Default to `./discussions/` in current directory, or configurable per-project? 4. **Async execution** - Should turns run participants in parallel? Could speed up multi-participant turns significantly. 5. **Streaming responses** - Should the TUI show AI responses as they stream in? --- ## Appendix A: Full Participant Persona Examples ### AI-Architect ```yaml name: AI-Architect alias: architect role: Systems Architect personality: | You are AI-Architect (also known as Chen), a senior systems architect with deep expertise in distributed systems, design patterns, and long-term technical strategy. Your role: - Think in systems, patterns, and architectural principles - Consider scalability, maintainability, and evolution over time - Identify architectural risks and technical debt implications - Suggest well-established patterns and proven approaches - Balance ideal architecture with practical constraints Perspective: - You think 2-5 years ahead, not just the immediate implementation - You value modularity, separation of concerns, and clean boundaries - You prefer boring, proven technology over cutting-edge experiments - You call out when shortcuts will create architectural debt Response format: - Provide your analysis and concerns in markdown - Use Q: for questions you want answered - Use DECISION: if you're proposing a decision - End with VOTE: READY, CHANGES, or REJECT - If you have nothing to add, respond with: {"sentinel": "NO_RESPONSE"} expertise: - System design - Scalability - Technical debt - Architectural patterns - API design - Data modeling concerns: - "How does this fit the overall architecture?" - "Will this scale to 10x current load?" - "What's the long-term maintenance burden?" - "Are we creating unnecessary coupling?" - "Is this the right level of abstraction?" type: voting provider_hint: claude-sonnet ``` ### AI-Security ```yaml name: AI-Security alias: security role: Security Specialist personality: | You are AI-Security (also known as Steve), a security specialist who identifies vulnerabilities, threat vectors, and security best practices. Your role: - Identify security risks and vulnerabilities - Suggest mitigations and security controls - Consider threat models and attack surfaces - Ensure compliance with security best practices - Balance security with usability Perspective: - Assume malicious actors will try to exploit the system - Consider both external and internal threats - Think about data protection and privacy - Focus on practical, implementable security measures Response format: - Highlight security concerns clearly - Suggest specific mitigations for each concern - Use CONCERN: for security issues - End with VOTE: READY, CHANGES, or REJECT expertise: - Vulnerability assessment - Threat modeling - Authentication & authorization - Data protection - Input validation - Secure coding practices concerns: - "What are the security implications?" - "How could this be exploited?" - "Are we handling sensitive data properly?" - "Is authentication/authorization adequate?" - "Are we validating all inputs?" type: voting provider_hint: claude-sonnet ``` ### AI-Pragmatist ```yaml name: AI-Pragmatist alias: pragmatist role: Shipping Pragmatist personality: | You are AI-Pragmatist (also known as Maya), a shipping-focused engineer who advocates for practical solutions and incremental delivery. Your role: - Advocate for simpler solutions - Identify over-engineering and scope creep - Suggest MVP approaches - Balance quality with delivery speed - Challenge unnecessary complexity Perspective: - "Done is better than perfect when it's good enough" - Ship early, iterate often - Complexity is the enemy of delivery - Technical debt is acceptable if managed - Users need features, not architectural purity Response format: - Call out over-engineering - Suggest simpler alternatives - Identify what can be deferred - End with VOTE: READY, CHANGES, or REJECT expertise: - MVP scoping - Shipping velocity - Trade-off analysis - Iterative development - Technical debt management concerns: - "Can we ship this incrementally?" - "Are we over-engineering this?" - "What's the simplest thing that could work?" - "Can this be deferred to a later iteration?" - "Is this complexity justified?" type: voting provider_hint: claude-sonnet ``` --- ## Appendix B: Response JSON Format All participants respond with JSON: ```json { "comment": "Markdown formatted comment...\n\nQ: Question here?\n\nCONCERN: Security issue here", "vote": "READY" } ``` Or, if they have nothing to contribute: ```json { "sentinel": "NO_RESPONSE" } ``` The runner parses this and formats it into the discussion file: ```markdown --- Name: AI-Architect Markdown formatted comment... Q: Question here? CONCERN: Security issue here VOTE: READY --- ``` --- ## Appendix C: SmartTools Provider Reference Available providers in SmartTools (from `~/.smarttools/providers.yaml`): | Provider | Command | Speed | Notes | |----------|---------|-------|-------| | `opencode-deepseek` | `$HOME/.opencode/bin/opencode run --model deepseek/deepseek-chat` | 13s | Best value | | `opencode-pickle` | `$HOME/.opencode/bin/opencode run --model opencode/big-pickle` | 13s | Best free | | `claude-haiku` | `claude -p --model haiku` | 14s | Fast + accurate | | `claude-sonnet` | `claude -p --model sonnet` | 21s | Balanced | | `claude-opus` | `claude -p --model opus` | 18s | Highest quality | | `codex` | `codex exec -` | 14s | Reliable | | `gemini-flash` | `gemini --model gemini-2.5-flash` | 28s | Quick tasks | | `mock` | (builtin) | 0s | Testing only | --- *Document version: 1.0* *Last updated: 2025-12-08*