orchestrated-discussions/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Build & Test Commands

```bash
# Install in development mode
pip install -e ".[dev]"

# Run all tests
pytest

# Run a single test file
pytest tests/test_markers.py

# Run a single test
pytest tests/test_markers.py::TestExtractVote::test_ready_vote

# Run with coverage
pytest --cov=discussions

# Test SmartTools directly (Unix philosophy - test tools independently)
cat examples/brainstorm_notification_system.md | discussion-parser | jq .
cat examples/brainstorm_notification_system.md | discussion-parser | discussion-vote-counter

# Find orphaned diagrams (not referenced by any discussion)
discussions cleanup

# Delete orphaned diagrams with confirmation
discussions cleanup --delete
```

## Architecture

Orchestrated Discussions is a multi-agent AI discussion orchestrator that manages structured conversations between AI personas with voting and phase-based workflows.

### Project Ecosystem

This is the middle layer of a three-project stack:
1. **SmartTools** - AI provider abstraction and tool execution (dependency)
2. **Orchestrated Discussions** (this) - Conversation orchestration
3. **CascadingDev** - Git-driven automation (depends on this)

---

## Architectural Philosophy: Unix Philosophy & Pipes and Filters

**This project strictly follows the Unix Philosophy.** This is non-negotiable.

### Core Principles

1. **Each tool does one thing well** - SmartTools are self-contained, complete units
2. **Tools communicate via stdin/stdout** - JSON flows through pipes
3. **Tools are composable** - Any tool's output can be another's input
4. **No tool knows about another's internals** - Only the interface (stdin/stdout/args)
5. **The discussion file is the source of truth** - Tools read and write to it

### Architecture Rules

**NEVER:**
- Import SmartTools internals (`from smarttools.providers import ...`)
- Duplicate logic that exists in a SmartTool
- Build "helper functions" when you should make a SmartTool
- Call AI providers directly - that's what SmartTools are for
- Put orchestration logic inside a tool (tools are stateless)

**ALWAYS:**
- Call SmartTools via subprocess: `subprocess.run(["tool-name"], input=data, ...)`
- Pass data through stdin, receive through stdout
- Keep Python layer thin - it only orchestrates, never implements
- Each SmartTool must be independently testable: `cat file | tool | jq .`
- Create a new SmartTool rather than adding complexity to existing code

### Anti-Pattern Examples

```python
# WRONG - Importing SmartTools internals, bypassing the tool
from smarttools.providers import call_provider
prompt = build_prompt_in_python(discussion)  # Logic that belongs in SmartTool
result = call_provider("claude", prompt)

# RIGHT - Calling the SmartTool via subprocess
result = subprocess.run(
    ["discussion-architect", "--callout", callout],
    input=discussion_content,
    capture_output=True,
    text=True
)
response = json.loads(result.stdout)
```

```python
# WRONG - Duplicating vote counting logic in Python
def count_votes(discussion):
    ready = sum(1 for v in votes if v == "READY")
    # ... reimplementing what discussion-vote-counter does

# RIGHT - Using the SmartTool
result = subprocess.run(
    ["discussion-vote-counter"],
    input=parser_output,
    capture_output=True,
    text=True
)
consensus = json.loads(result.stdout)
```

### Why This Matters

Linux follows these principles and is arguably the most powerful and stable software system ever created. Benefits:

- **Debuggability**: Test any tool in isolation: `cat test.md | discussion-parser | jq .`
- **Flexibility**: Swap implementations without changing orchestration
- **Reliability**: Each tool can be hardened independently
- **Composability**: Build new workflows by combining existing tools
- **Maintainability**: Change one tool without breaking others

### The Python Layer's Role

The Python code in this project (`runner.py`, `cli.py`, `ui/`) is **orchestration only**:

- Decide which tools to call and in what order
- Pass data between tools (pipe stdout to stdin)
- Handle errors and retries
- Provide user interface

It should **never** contain:
- AI prompt construction (that's in SmartTool configs)
- Vote counting logic (that's `discussion-vote-counter`)
- Response parsing logic (that's in each SmartTool's code steps)
- Discussion format knowledge (that's `discussion-parser`)

---

### Key Design Decision: Participants ARE SmartTools

Participants are implemented as SmartTools, not a separate system. Each participant lives in `~/.smarttools/discussion-{alias}/config.yaml` and can be:
- Invoked directly for testing: `cat discussion.md | discussion-architect --callout "..."`
- Edited via SmartTools TUI for prompt debugging
- Created via `discussions participants add` wizard (generates SmartTool config)

This means:
- No duplicate participant definition system
- Full SmartTools features available (multi-step pipelines, code steps, provider fallbacks)
- Independent testing and debugging of each participant

### Core Abstractions

- **Discussion** (`discussion.py`): A markdown file with metadata headers, context, and comment blocks. Append-only - content grows but never shrinks. State stored in HTML comments (`<!-- Key: value -->`).

- **Participant**: A SmartTool in `~/.smarttools/discussion-{alias}/`. Discovered by naming convention. Has personality prompt, expertise metadata, and vote behavior. Two types: `voting` and `background`.

- **Markers** (`markers.py`): Structured annotations parsed from comment text:
  - `VOTE: READY|CHANGES|REJECT`
  - `Q:` / `QUESTION:` - Questions
  - `TODO:` / `ACTION:` - Action items
  - `DECISION:` - Recorded decisions
  - `CONCERN:` - Raised concerns
  - `DIAGRAM:` - Diagram file references (e.g., `DIAGRAM: diagrams/flow.puml`)
  - `@alias` - Mentions

- **Voting** (`voting.py`): Consensus calculation with configurable thresholds.

- **Runner** (`runner.py`): Invokes participant SmartTools via subprocess, parses JSON responses, appends to discussion file.

### Discussion File Format

```markdown
<!-- DISCUSSION -->
<!-- Title: Feature Name -->
<!-- Phase: initial_feedback -->
<!-- Status: OPEN -->
<!-- Template: feature -->
<!-- Participants: architect, security, pragmatist -->

# Title

## Context
...

---

Name: AI-Architect
Comment content with markers...

VOTE: CHANGES

---
```

Comment blocks are delimited by `---` and start with `Name: AuthorName`.

### Template System

Templates define phase-specific behavior for discussions. Stored in `templates/` as YAML files.

```yaml
# templates/feature.yaml
phases:
  initial_feedback:
    goal: Gather diverse perspectives
    instructions: |
      - Focus on feasibility and risks
      - Raise blocking issues early      
    voting: false
    next_phase: detailed_review
```

**How it works:**
1. Discussion references template via `<!-- Template: feature -->`
2. Participant SmartTools read the template file to get phase context
3. AI prompts include phase goal and instructions for context-aware responses

Participants use `--templates-dir` argument (default: `templates`) to locate template files.

### Consensus Logic

- `VotingConfig.threshold_ready` (default 0.67): Fraction of READY votes needed
- `VotingConfig.threshold_reject` (default 0.01): Any REJECT blocks by default
- `VotingConfig.human_required` (default True): Needs human READY vote
- Human participants detected by name NOT starting with `ai_`, `ai-`, `bot_`, `bot-`

### SmartTools Integration

```
Orchestrated Discussions (thin orchestration layer)
    │ invokes via subprocess
    ▼
SmartTools (discussion-* tools)
    │
    ├── Utility tools (code-only):
    │   - discussion-parser      → JSON structured data
    │   - discussion-validator   → validation results
    │
    ├── Orchestration tools (code-only):
    │   - discussion-vote-counter     → consensus calculation
    │   - discussion-mention-router   → participant routing
    │   - discussion-status-promoter  → status transitions
    │   - discussion-turn-appender    → append responses
    │   - discussion-config           → modify metadata
    │
    ├── Utility tools (AI):
    │   - discussion-summarizer  → .sum.md files
    │
    └── Participant tools (AI):
        - discussion-moderator, discussion-architect, etc.
            │ uses providers
            ▼
        AI Providers (claude, codex, gemini, etc.)
```

**Everything is a SmartTool** - parsing, validation, orchestration, summarization, and participants. This means:
- All components testable independently: `cat discussion.md | discussion-parser | jq .`
- Full pipelines runnable manually: `cat d.md | discussion-parser | discussion-vote-counter | jq .`
- Debuggable via SmartTools TUI
- Composable: tools can call other tools

### Manual Turn Execution

```bash
# Parse → route → call participants → append → count votes → promote status
DISCUSSION="feature.md"
STATE=$(cat "$DISCUSSION" | discussion-parser)
ROUTING=$(echo "$STATE" | discussion-mention-router)
# ... call each participant, collect responses ...
# ... append responses, count votes, promote status ...
```

See `scripts/run-turn.sh` for a complete example.

### Project Files

- `docs/DESIGN.md` - Full architecture and SmartTool specifications
- `docs/IMPLEMENTATION.md` - Phased implementation plan
- `smarttools/` - Bundled SmartTool configs (copied to ~/.smarttools/ on install)
- `templates/` - Discussion workflow templates (phase definitions)
- `examples/` - Example discussion files for testing
- `scripts/run-turn.sh` - Manual turn orchestration script

Participants respond with JSON: `{"comment": "...", "vote": "READY|CHANGES|REJECT|null"}`
or sentinel: `{"sentinel": "NO_RESPONSE"}`

### UI Options

Two UI implementations available:
- **GUI** (default): Dear PyGui-based, native image viewing for diagrams, read-aloud buttons
- **TUI**: urwid-based, text-only terminal interface, read-aloud buttons

Both UIs include:
- "Read" button on each comment for text-to-speech (requires `~/.smarttools/read-aloud/`)
- "Artifact" button in comment dialog for creating diagrams/visuals

### Artifact Editor Integration

Both UIs integrate with the standalone Artifact Editor (`~/PycharmProjects/artifact-editor`) for creating visual artifacts:

**GUI Integration:**
- "Add Artifact" button in the comment dialog
- "New Artifact" button in the diagrams panel
- Launches artifact-editor, waits for save, captures actual file path
- Automatically adds `DIAGRAM: path/to/file` marker to comment

**TUI Integration:**
- "Artifact" button in the comment input widget
- If `$DISPLAY` is available: launches artifact-editor GUI
- If headless (SSH): falls back to `$EDITOR` (nano/vim) for text editing
- Diagram reference added to comment on save

**Integration Protocol:**
```bash
# Artifact editor outputs on save:
ARTIFACT_SAVED:/absolute/path/to/file.svg

# Parent app parses this to get actual saved path
# (handles format changes - user may switch from .puml to .svg)
```

```bash
# Launch GUI (default)
discussions ui

# Launch TUI
discussions ui --tui

# Or via module
python -m src.discussions.ui           # GUI
python -m src.discussions.ui --tui     # TUI
```

### Keyboard Shortcuts (GUI)

| Key | Action |
|-----|--------|
| `Q` | Quit |
| `R` | Refresh |
| `T` | Run turn |
| `C` | Add comment |
| `D` | View diagrams |
| `Esc` | Close dialogs |

### Keyboard Shortcuts (TUI)

| Key | Action |
|-----|--------|
| `q` | Quit |
| `r` | Run turn (invoke participants) |
| `d` | View diagrams (ASCII preview) |
| `↑/↓` | Navigate |
| `Enter` | Select |
| `Esc` | Close dialogs |

### SmartTools Arguments

Participant SmartTools accept these arguments:
- `--callout` - Specific question or @mention context
- `--templates-dir` - Path to templates directory (default: `templates`)
- `--diagrams-dir` - Path to save diagrams (default: `diagrams`)
- `--log-file` - Path to log file for progress updates (used by TUI for parallel execution)

### Source Structure

```
src/discussions/
├── cli.py           # CLI entry point (discussions command)
├── discussion.py    # Discussion model, file I/O
├── participant.py   # Participant discovery from ~/.smarttools/discussion-*
├── markers.py       # Marker parsing (VOTE:, Q:, TODO:, CONCERN:, etc.)
├── voting.py        # Consensus calculation
├── runner.py        # Turn orchestration (calls SmartTools via subprocess)
└── ui/
    ├── __init__.py
    ├── __main__.py  # Module entry point
    ├── gui.py       # Dear PyGui interface (default)
    ├── tui.py       # urwid terminal interface
    └── widgets.py   # Shared UI components
```