# Orchestrated Discussions - Design Document

## Implementation Status

| Component | Status | Notes |
|-----------|--------|-------|
| Core library | Complete | discussion.py, markers.py, voting.py, participant.py |
| Runner | Complete | Pipeline-based turn execution |
| CLI | Complete | All commands implemented |
| GUI | Complete | Dear PyGui interface |
| TUI | Complete | urwid terminal interface |
| Utility SmartTools | Complete | parser, validator, summarizer |
| Orchestration SmartTools | Complete | vote-counter, mention-router, status-promoter, turn-appender, config |
| Participant SmartTools | Complete | architect, security, pragmatist, moderator, diagram-editor |
| Templates | Complete | feature.yaml, brainstorm.yaml |

---

## Architectural Philosophy

**This project strictly follows the Unix Philosophy:**

1. **Each tool does one thing well** - SmartTools are self-contained, complete units
2. **Tools communicate via stdin/stdout** - JSON flows through pipes
3. **Tools are composable** - Any tool's output can be another's input
4. **The Python layer is thin orchestration only** - It calls tools, never reimplements their logic

See `CLAUDE.md` for detailed architectural rules and anti-patterns.

---

## Overview

**Orchestrated Discussions** is a Python library and CLI for running structured, multi-participant discussions between AI personas and human contributors. It provides turn-based orchestration, role-driven responses, and voting-based consensus checks across phases.

### Project Context

This project is part of a three-project ecosystem:

1. **SmartTools** - Foundation Layer
   - AI provider abstraction, prompt execution pipelines, YAML-defined tools

2. **Orchestrated Discussions** - Conversation Layer (this project)
   - Structured, multi-participant discussions with AI personas and humans
   - Phases, turns, consensus voting, append-only discussion files

3. **CascadingDev** - Automation & Integration Layer
   - Uses Orchestrated Discussions for feature discussions, code reviews
   - Integrates with Git hooks and cascading rules

### Design Goals

1. **Workflow independence** - Operates independently of Git/CascadingDev
2. **SmartTools as the provider layer** - All AI inference through SmartTools
3. **Extensible participants** - Support AI personas and human participants
4. **Multiple interfaces** - CLI, Python API, TUI/GUI
5. **Append-only discussion files** - Human-readable, Git-friendly
6. **Portable, self-contained state** - All state in the markdown file

---

## Architecture

### Dependency Graph

```
┌────────────────────────────────────────────┐
│              User Applications             │
│   (CascadingDev, automation scripts, UI)   │
└─────────────────────┬──────────────────────┘
                      │
┌─────────────────────▼──────────────────────┐
│           Orchestrated Discussions         │
│  - Append-only discussion files (Markdown) │
│  - Phase and turn orchestration            │
│  - Multi-participant workflows (AI + human)│
│  - Voting + consensus logic                │
│  - @mention routing and participant logic  │
│  - TUI/GUI for real-time display           │
└─────────────────────┬──────────────────────┘
                      │
┌─────────────────────▼──────────────────────┐
│                 SmartTools                 │
│  - AI provider abstraction                 │
│  - Inference dispatch + fallback chains    │
│  - YAML-defined tool pipelines             │
│  - Streaming / non-streaming execution     │
└────────────────────────────────────────────┘
```

### Module Structure (Actual)

```
orchestrated-discussions/
├── src/discussions/
│   ├── __init__.py           # Package exports, version
│   ├── cli.py                # CLI entry point (argparse)
│   ├── discussion.py         # Discussion model, file I/O
│   ├── participant.py        # Participant discovery from ~/.smarttools/
│   ├── markers.py            # Marker parsing (VOTE, Q, TODO, etc.)
│   ├── voting.py             # Voting + consensus calculation
│   ├── runner.py             # Turn orchestration (calls SmartTools)
│   └── ui/
│       ├── __init__.py       # UI module exports
│       ├── __main__.py       # Module entry point
│       ├── gui.py            # Dear PyGui interface
│       ├── tui.py            # urwid terminal interface
│       └── widgets.py        # Shared UI components
│
├── smarttools/               # Bundled SmartTool configs
│   ├── discussion-parser/
│   ├── discussion-validator/
│   ├── discussion-summarizer/
│   ├── discussion-vote-counter/
│   ├── discussion-mention-router/
│   ├── discussion-status-promoter/
│   ├── discussion-turn-appender/
│   ├── discussion-config/
│   ├── discussion-moderator/
│   ├── discussion-architect/
│   ├── discussion-security/
│   ├── discussion-pragmatist/
│   ├── discussion-diagram-editor/
│   └── puml-validator/
│
├── templates/                # Discussion workflow templates
│   ├── feature.yaml          # 3-phase feature workflow
│   └── brainstorm.yaml       # 6-phase ideation workflow
│
├── examples/                 # Example discussion files
│   ├── feature_discussion.md
│   ├── voted_discussion.md
│   ├── brainstorm_notification_system.md
│   ├── brainstorm_sketch_test.md
│   └── ai-enabled-home-lab-productivity-and-development-tool.md
│
├── scripts/
│   └── run-turn.sh           # Manual turn orchestration script
│
├── tests/
│   ├── test_markers.py       # Marker parsing tests
│   └── test_voting.py        # Voting calculation tests
│
└── docs/
    ├── DESIGN.md             # This document
    ├── IMPLEMENTATION.md     # Implementation plan
    └── PIPELINE_SCHEMA.md    # Pipeline configuration docs
```

---

## Core Concepts

### 1. Discussion

A Discussion is an append-only, self-contained markdown file with:

- **Metadata**: Title, phase, status, template, participants (in HTML comments)
- **Context**: Description of what's being discussed
- **Comments**: Appended blocks from participants, chronological
- **Markers**: Structured annotations (VOTE, Q, TODO, DECISION, CONCERN, DIAGRAM)
- **@mentions**: Inline references to participants

#### Discussion File Format

```markdown
<!-- DISCUSSION -->
<!-- Title: Feature X Implementation -->
<!-- Phase: initial_feedback -->
<!-- Status: OPEN -->
<!-- Created: 2025-12-08T10:30:00Z -->
<!-- Template: feature -->
<!-- Participants: architect, security, pragmatist -->

# Feature X Implementation

## Context
We need to implement feature X that allows users to...

---

Name: AI-Architect
Looking at this from a systems perspective...

Q: Have we considered using the adapter pattern here?
@pragmatist What do you think about complexity?

VOTE: CHANGES

---

Name: Human
I agree with the architect's concerns.
@security — can you evaluate the session-handling implications?

VOTE: CHANGES

---
```

### 2. Participant

Participants are SmartTools with the naming convention `discussion-{alias}`:

- **Invoked directly** for testing: `cat discussion.md | discussion-architect --callout "..."`
- **Edited via SmartTools TUI** for prompt debugging
- **Discovered automatically** by scanning `~/.smarttools/discussion-*`

#### Participant Types

| Type | Behavior |
|------|----------|
| `voting` | Casts VOTE: READY/CHANGES/REJECT |
| `background` | Provides input without voting |

#### Input/Output Contract

**Input** (via stdin):
- Full discussion markdown content

**Arguments**:
- `--callout` - Specific question or @mention context
- `--templates-dir` - Path to templates directory

**Output** (JSON):
```json
{
  "comment": "Markdown analysis with markers...",
  "vote": "READY|CHANGES|REJECT|null"
}
```

Or sentinel for no response:
```json
{"sentinel": "NO_RESPONSE"}
```

### 3. Markers

Structured annotations parsed from comment text:

| Marker | Pattern | Description |
|--------|---------|-------------|
| `VOTE:` | `VOTE: READY\|CHANGES\|REJECT` | Participant's vote |
| `Q:` | `Q: question text` | Question |
| `TODO:` | `TODO: action item` | Action item |
| `DECISION:` | `DECISION: we will...` | Recorded decision |
| `CONCERN:` | `CONCERN: issue...` | Raised concern |
| `DIAGRAM:` | `DIAGRAM: path/to/file.puml` | Diagram reference |
| `@alias` | `@architect` | Mention participant |

### 4. Phases

Phases define the stages of a discussion. Defined in template YAML files:

```yaml
phases:
  initial_feedback:
    goal: Gather diverse perspectives
    instructions: |
      - Focus on feasibility and risks
      - Raise blocking issues early
    voting: false
    next_phase: detailed_review

  consensus_vote:
    goal: Reach agreement
    instructions: |
      - Vote READY if all concerns addressed
      - Vote CHANGES if issues remain
    voting: true
    threshold_ready: 0.67
    human_required: true
    next_phase: null
```

### 5. Templates

Templates define workflow structure for different discussion types:

#### Feature Template (`templates/feature.yaml`)

| Phase | Goal | Voting |
|-------|------|--------|
| `initial_feedback` | Gather diverse perspectives | No |
| `detailed_review` | Deep dive into implementation | No |
| `consensus_vote` | Reach agreement on approach | Yes (67%) |

#### Brainstorm Template (`templates/brainstorm.yaml`)

| Phase | Goal | Voting |
|-------|------|--------|
| `seed` | Frame the problem | No |
| `diverge` | Generate ideas freely | No |
| `cluster` | Group into themes | Yes (50%) |
| `sketch` | Create rough diagrams | No |
| `reality_check` | Ground in reality | No |
| `decide` | Commit to approach | Yes (67%) |

### 6. Voting

**Vote Types**:
- `READY` - Approve, no blocking concerns
- `CHANGES` - Conditional approval, needs modifications
- `REJECT` - Block, fundamental issues

**Consensus Rules** (configurable per phase):
- `threshold_ready`: Fraction of READY votes needed (default: 0.67)
- `threshold_reject`: Any REJECT blocks by default (0.01)
- `human_required`: Whether human approval is mandatory

**Human Detection**: Participants NOT starting with `ai_`, `ai-`, `bot_`, `bot-` are considered human.

### 7. Turns

A Turn is one round of participant responses. The runner:

1. Loads discussion and parses current state
2. Determines who should respond (mentions or all)
3. Calls each participant SmartTool in parallel
4. Appends responses to discussion file
5. Counts votes and checks consensus
6. Promotes status if consensus reached

---

## SmartTools Integration

### Tool Pipeline Architecture

```
┌─────────────────────────────────────────────────────────┐
│              Orchestrated Discussions                    │
│  - Thin orchestration layer (runner.py)                 │
│  - File I/O, turn sequencing                            │
│  - Invokes SmartTools for ALL logic                     │
└─────────────────────┬───────────────────────────────────┘
                      │ invokes SmartTools via subprocess
                      ▼
┌─────────────────────────────────────────────────────────┐
│              SmartTools (discussion-* tools)             │
│                                                         │
│  Utility tools:                                         │
│  - discussion-parser      (extract structured data)     │
│  - discussion-validator   (check format, find issues)   │
│  - discussion-summarizer  (generate .sum.md files)      │
│                                                         │
│  Orchestration tools:                                   │
│  - discussion-vote-counter     (count votes)            │
│  - discussion-mention-router   (route @mentions)        │
│  - discussion-status-promoter  (status transitions)     │
│  - discussion-turn-appender    (append responses)       │
│  - discussion-config           (modify metadata)        │
│                                                         │
│  Participant tools:                                     │
│  - discussion-architect   (systems thinking)            │
│  - discussion-security    (threat modeling)             │
│  - discussion-pragmatist  (shipping focus)              │
│  - discussion-moderator   (facilitation)                │
│  - discussion-diagram-editor (PlantUML diagrams)        │
└─────────────────────┬───────────────────────────────────┘
                      │ uses providers (for AI steps)
                      ▼
┌─────────────────────────────────────────────────────────┐
│              SmartTools Providers                        │
│  claude-sonnet, claude-haiku, opencode-deepseek, etc.   │
└─────────────────────────────────────────────────────────┘
```

### Turn Pipeline Configuration

Templates can define custom pipelines using a variable-based schema:

```yaml
turn_pipeline:
  steps:
    - tool: discussion-parser
      input: $discussion
      output: $parsed

    - tool: discussion-mention-router
      input: $parsed
      output: $routing
      when: not $participants_specified
      args:
        --default-participants: $participants_csv

    - tool: discussion-{participant}
      for_each: $participants_to_call
      parallel: true
      input: $discussion
      output: $responses[]
      args:
        --callout: $callout
        --templates-dir: $templates_dir

    - tool: discussion-turn-appender
      input: $discussion
      output: $discussion
      args:
        --responses-json: $responses_json

    - tool: discussion-vote-counter
      input: $parsed
      output: $votes
      when: $phase_voting

    - tool: discussion-status-promoter
      input: $votes
      output: $promotion
      when: $phase_voting
      args:
        --current-status: $status
        --current-phase: $phase
```

### Testing Tools Independently

```bash
# Parse discussion to JSON
cat discussion.md | discussion-parser | jq .

# Count votes
cat discussion.md | discussion-parser | discussion-vote-counter

# Route mentions
cat discussion.md | discussion-parser | discussion-mention-router

# Full pipeline manually
./scripts/run-turn.sh discussion.md
```

---

## Bundled SmartTools

### Utility Tools (Code-only)

#### discussion-parser

Extracts structured data from discussion markdown.

**Input**: Discussion markdown (stdin)
**Output**: JSON with metadata, comments, votes, markers

```json
{
  "metadata": {
    "title": "Feature X",
    "phase": "initial_feedback",
    "status": "OPEN",
    "template": "feature",
    "participants": ["architect", "security"]
  },
  "comments": [
    {"author": "AI-Architect", "body": "...", "vote": "CHANGES"}
  ],
  "vote_summary": {"READY": 1, "CHANGES": 2, "REJECT": 0, "total": 3},
  "questions": ["Have we considered...?"],
  "concerns": ["Security implications"],
  "mentions": ["security", "pragmatist"]
}
```

#### discussion-validator

Validates discussion format and identifies issues.

**Input**: Discussion markdown (stdin)
**Output**: JSON with validation results

```json
{
  "valid": true,
  "issues": [],
  "warnings": ["Pending responses from: security"]
}
```

#### discussion-summarizer

Generates a summary of the discussion (AI-powered).

**Input**: Discussion markdown (stdin)
**Output**: Markdown summary

### Orchestration Tools (Code-only)

#### discussion-vote-counter

Counts votes and determines consensus status.

**Input**: Parser JSON output (stdin)
**Arguments**:
- `--threshold-ready` (default: 0.67)
- `--threshold-reject` (default: 0.01)
- `--human-required` (default: true)

**Output**: JSON with vote counts and consensus status

```json
{
  "votes": {"AI-Architect": "CHANGES", "Human": "READY"},
  "vote_summary": {"READY": 1, "CHANGES": 1, "REJECT": 0, "total": 2},
  "consensus": {
    "reached": false,
    "reason": "Need 1 more READY votes"
  }
}
```

#### discussion-mention-router

Determines which participants should respond based on @mentions.

**Input**: Parser JSON output (stdin)
**Arguments**:
- `--default-participants` (default: "architect,security,pragmatist")

**Output**: JSON with routing information

```json
{
  "participants_to_call": ["security", "pragmatist"],
  "callouts": {"security": "", "pragmatist": ""},
  "pending_mentions": ["security"]
}
```

#### discussion-status-promoter

Determines if discussion status should change based on consensus.

**Input**: Vote-counter JSON output (stdin)
**Arguments**:
- `--current-status`
- `--current-phase`

**Output**: JSON with promotion decision

```json
{
  "current_status": "OPEN",
  "new_status": "READY_FOR_DESIGN",
  "should_promote": true,
  "transition_reason": "Consensus reached"
}
```

#### discussion-turn-appender

Appends participant responses to discussion markdown.

**Input**: Discussion markdown + `---RESPONSES---` + JSON array (stdin)
**Output**: Updated discussion markdown

#### discussion-config

Modifies discussion metadata (phase, status, participants).

**Arguments**:
- `--set-phase`
- `--set-status`
- `--add-participant`
- `--insert-marker`

### Participant Tools (AI)

All participant tools follow the same contract:

**Input**: Discussion markdown (stdin)
**Arguments**: `--callout`, `--templates-dir`
**Output**: JSON with `comment` and `vote` fields

| Tool | Alias | Perspective |
|------|-------|-------------|
| `discussion-architect` | `@architect` | Systems thinking, scalability, patterns |
| `discussion-security` | `@security` | Threat modeling, vulnerabilities |
| `discussion-pragmatist` | `@pragmatist` | MVP focus, shipping velocity |
| `discussion-moderator` | `@moderator` | Facilitation, phase management |
| `discussion-diagram-editor` | `@diagram-editor` | PlantUML diagram creation |

---

## CLI Interface

### Commands

```bash
# Create a new discussion
discussions new "Feature X" --template feature
discussions new "Brainstorm" --template brainstorm

# Show discussion status
discussions status feature-x.md

# Run a turn
discussions turn feature-x.md @architect @security
discussions turn feature-x.md  # Uses mention router

# Add human comment
discussions comment feature-x.md "I agree" --vote READY

# Advance phase manually
discussions advance feature-x.md --phase detailed_review

# List available participants
discussions participants

# Launch UI
discussions ui             # GUI (default)
discussions ui --tui       # Terminal UI
```

---

## Python API

```python
from discussions import Discussion
from discussions.runner import run_pipeline_turn

# Load or create discussion
discussion = Discussion.load("feature-x.md")
discussion = Discussion.create(
    path="new-feature.md",
    title="New Feature",
    template="feature",
    context="We need to..."
)

# Access state
print(discussion.title)           # "New Feature"
print(discussion.phase)           # "initial_feedback"
print(discussion.status)          # "OPEN"
print(discussion.get_votes())     # {"AI-Architect": "READY", ...}
print(discussion.get_questions()) # [Question(...), ...]

# Add comment
discussion.add_comment(
    author="Human",
    text="I agree with this approach.",
    vote="READY"
)
discussion.save()

# Check consensus
result = discussion.check_consensus()
if result.reached:
    print(f"Consensus: {result.outcome}")
else:
    print(f"Blocked: {result.reason}")

# Run turn via SmartTools pipeline
from pathlib import Path
result = run_pipeline_turn(
    discussion_path=Path("feature-x.md"),
    participants=["architect", "security"],
    callout="Please review the proposal",
    verbose=True
)
print(f"Responses: {result.successful_count}")
```

---

## User Interfaces

### GUI (Dear PyGui)

- **Native image viewing** for PlantUML diagrams
- **Split-pane diagram viewer** with source and rendered preview
- **Tabbed interface** for discussions, diagrams, and output

Launch: `discussions ui` or `python -m src.discussions.ui`

### TUI (urwid)

- **ASCII diagram preview** via `plantuml -tutxt`
- **Works over SSH** without X11
- **Lightweight** terminal interface

Launch: `discussions ui --tui` or `python -m src.discussions.ui --tui`

### Keyboard Shortcuts

| Key | GUI | TUI |
|-----|-----|-----|
| Quit | `Q` | `q` |
| Refresh | `R` | - |
| Run turn | `T` | `r` |
| Add comment | `C` | - |
| View diagrams | `D` | `d` |
| Close dialog | `Esc` | `Esc` |

---

## Configuration

### Project Configuration

Create `discussions.yaml` in project root:

```yaml
default_participants:
  - architect
  - security
  - pragmatist

default_template: feature

consensus:
  threshold_ready: 0.67
  threshold_reject: 0.01
  human_required: true

output:
  directory: ./discussions
```

### Participant Discovery

Participants are discovered by scanning `~/.smarttools/` for tools matching `discussion-*`:

```bash
~/.smarttools/
├── discussion-architect/config.yaml    # @architect
├── discussion-security/config.yaml     # @security
├── discussion-pragmatist/config.yaml   # @pragmatist
├── discussion-moderator/config.yaml    # @moderator
└── discussion-diagram-editor/config.yaml  # @diagram-editor
```

List discovered participants:
```bash
discussions participants
```

---

## Testing

### Unit Tests

```bash
# Run all tests
pytest

# Run specific test file
pytest tests/test_markers.py

# Run with coverage
pytest --cov=discussions
```

### SmartTools Integration Tests

```bash
# Test parser
cat examples/feature_discussion.md | discussion-parser | jq .

# Test full pipeline
./scripts/run-turn.sh examples/feature_discussion.md
```

---

## Migration from CascadingDev

Once stable, CascadingDev will:

1. **Remove**: Discussion orchestration code (schema.py, workflow.py)
2. **Remove**: Voting agent implementations
3. **Add**: `orchestrated-discussions` as dependency
4. **Update**: Git hooks to use `discussions` CLI

---

## UML Diagrams

Visual documentation is available in `docs/diagrams/`:

| Diagram | Description |
|---------|-------------|
| [turn-execution-sequence.puml](diagrams/turn-execution-sequence.puml) | Sequence diagram showing how a discussion turn flows through the system |
| [architecture-component.puml](diagrams/architecture-component.puml) | Component diagram showing the overall system architecture |
| [core-class-diagram.puml](diagrams/core-class-diagram.puml) | Class diagram of the Python module structure |
| [smarttools-pipeline-flow.puml](diagrams/smarttools-pipeline-flow.puml) | Activity diagram showing data flow through SmartTools pipeline |
| [discussion-state-machine.puml](diagrams/discussion-state-machine.puml) | State machine showing discussion lifecycle and status transitions |

To render diagrams:
```bash
# Single diagram
plantuml docs/diagrams/turn-execution-sequence.puml

# All diagrams
plantuml docs/diagrams/*.puml

# ASCII preview
plantuml -tutxt docs/diagrams/turn-execution-sequence.puml
```

---

*Document version: 2.0*
*Last updated: 2025-12-16*