orchestrated-discussions/docs/PIPELINE_SCHEMA.md

Turn Pipeline Schema

Overview

Each template can define a turn_pipeline that specifies which SmartTools to run and how to pipe data between them. The runner is thin orchestration - it just executes tools and pipes data, with no business logic.

Design Philosophy

The pipeline should mirror what you could do manually in a shell:

# Parse the discussion
cat discussion.md | discussion-parser > parsed.json

# Route mentions
cat parsed.json | discussion-mention-router > routing.json

# Call each participant (could be parallel)
cat discussion.md | discussion-architect --callout "Review" > resp_architect.json
cat discussion.md | discussion-security --callout "Review" > resp_security.json

# Append responses
cat discussion.md | discussion-turn-appender --responses '[...]' > updated.md

# Count votes (if voting phase)
cat updated.md | discussion-parser | discussion-vote-counter > votes.json

Schema

turn_pipeline:
  steps:
    - tool: <tool-name>           # Required: SmartTool to execute
      input: <variable>           # What to pipe to stdin (default: $discussion)
      output: <variable>          # Where to store stdout (default: discarded)
      args:                       # CLI arguments
        --arg-name: <value>       # Can reference variables: $varname
      when: <condition>           # Optional: condition to run this step
      for_each: <variable>        # Optional: iterate over array
      parallel: <bool>            # Optional: run for_each in parallel

Variables

Variables are referenced with $ prefix:

Built-in Variables

• $discussion - Current discussion markdown content (updated as pipeline runs)
• $discussion_path - Path to the discussion file
• $callout - Callout/question passed to the turn
• $participants - List of participant aliases from discussion metadata
• $templates_dir - Path to templates directory

Step Output Variables

Each step can capture its output:

- tool: discussion-parser
  input: $discussion
  output: $parsed        # Now $parsed contains the JSON output

Accessing JSON Fields

For JSON outputs, access fields with dot notation:

- tool: discussion-mention-router
  input: $parsed
  output: $routing

- tool: discussion-{participant}
  for_each: $routing.participants_to_call   # Iterate over array in JSON

Array Collection

When using for_each, outputs are collected into an array:

- tool: discussion-{participant}
  for_each: $routing.participants_to_call
  output: $responses[]    # [] means append to array

Conditions

• always - Always run (default)
• $variable - Run if variable is truthy
• $variable.field - Run if JSON field is truthy
• not $variable - Run if variable is falsy

Examples:

- tool: discussion-vote-counter
  when: $phase.voting                    # Only if phase has voting enabled

- tool: discussion-mention-router
  when: not $participants_specified      # Only if no participants given

Complete Example

turn_pipeline:
  steps:
    # Step 1: Parse discussion to get metadata
    - tool: discussion-parser
      input: $discussion
      output: $parsed

    # Step 2: Route mentions (optional - skip if participants specified)
    - tool: discussion-mention-router
      input: $parsed
      output: $routing
      when: not $participants_specified
      args:
        --default-participants: $participants

    # Step 3: Call each participant
    - tool: discussion-{participant}
      for_each: $routing.participants_to_call
      parallel: true
      input: $discussion
      output: $responses[]
      args:
        --callout: $callout
        --templates-dir: $templates_dir

    # Step 4: Append responses to discussion
    - tool: discussion-turn-appender
      input: $discussion
      output: $discussion              # Update the discussion variable
      args:
        --responses-json: $responses

    # Step 5: Count votes (only in voting phases)
    - tool: discussion-parser
      input: $discussion
      output: $updated_parsed
      when: $phase.voting

    - tool: discussion-vote-counter
      input: $updated_parsed
      output: $votes
      when: $phase.voting
      args:
        --threshold-ready: $phase.threshold_ready
        --human-required: $phase.human_required

    # Step 6: Check status promotion
    - tool: discussion-status-promoter
      input: $votes
      output: $promotion
      when: $phase.voting
      args:
        --current-status: $parsed.metadata.status
        --current-phase: $parsed.metadata.phase

For Each Expansion

When for_each is used, the step runs once per item:

- tool: discussion-{participant}
  for_each: $routing.participants_to_call   # e.g., ["architect", "security"]
  input: $discussion
  output: $responses[]

This expands to:

cat discussion.md | discussion-architect > resp_0.json
cat discussion.md | discussion-security > resp_1.json
# $responses = [resp_0.json contents, resp_1.json contents]

With parallel: true, these run concurrently.

Special Tool Patterns

{participant} Substitution

The pattern {participant} in tool names is replaced with the current iteration value:

- tool: discussion-{participant}
  for_each: ["architect", "security"]

Becomes calls to discussion-architect and discussion-security.

Minimal Example

A bare-bones pipeline that just calls participants and appends:

turn_pipeline:
  steps:
    - tool: discussion-{participant}
      for_each: $participants
      parallel: true
      input: $discussion
      output: $responses[]

    - tool: discussion-turn-appender
      input: $discussion
      output: $discussion
      args:
        --responses-json: $responses

Phase Artifacts

Phases can define artifacts to enable collaboration on diagrams, sketches, and models. Participants read this config to know what formats to output and where to save files.

Artifact Schema

phases:
  sketch:
    goal: Rough out the selected idea
    artifacts:
      formats:                    # List of accepted formats
        - plantuml                # PlantUML diagrams (.puml)
        - svg                     # SVG vector graphics (.svg)
        - mermaid                 # Mermaid diagrams (.mmd)
        - openscad                # OpenSCAD 3D models (.scad)
        - solidpython             # SolidPython 3D models (.py)
      output_dir: diagrams/       # Where to save artifacts
      file_pattern: "{title}_{participant}_{n}"  # Naming pattern
    instructions: |
      - Create diagrams in any supported format
      - Review existing artifacts in the output directory
      - Build on or modify others' work as needed      

Supported Formats

Format	Extension	Use Case
plantuml	.puml	Sequence, component, class, state diagrams
mermaid	.mmd	Flowcharts, sequence, ER, Gantt charts
svg	.svg	UI mockups, icons, 2D layouts, sketches
openscad	.scad	3D models, mechanical parts, enclosures
solidpython	.py	Programmatic 3D models (Python-based)
dot	.dot	Graph visualizations (Graphviz)
ascii	.txt	Simple inline text diagrams

How Participants Use Artifacts

Participants receive artifact config via the --artifacts-json argument:

{
  "formats": ["plantuml", "svg"],
  "output_dir": "diagrams/",
  "file_pattern": "{title}_{participant}_{n}"
}

They then:

1. Know what formats are acceptable for this phase
2. Save artifacts to the correct directory
3. Reference artifacts in comments with ARTIFACT: path/to/file.ext
4. Can read existing artifacts in output_dir to build upon them

Response Format with Artifacts

Participants output artifacts in their JSON response:

{
  "comment": "Here's my proposed architecture...\n\nARTIFACT: diagrams/auth-flow_architect_1.puml",
  "vote": null,
  "artifacts": [
    {
      "format": "plantuml",
      "filename": "auth-flow_architect_1.puml",
      "content": "@startuml\nUser -> Auth: login\nAuth -> DB: validate\n@enduml"
    }
  ]
}

Example: Hardware Design Template

name: hardware_design
description: Physical product design workflow

phases:
  requirements:
    goal: Define physical constraints and requirements
    artifacts: null  # No artifacts in this phase
    voting: false

  enclosure_design:
    goal: Design the physical enclosure
    artifacts:
      formats: [openscad, solidpython]
      output_dir: models/
    instructions: |
      - Create 3D models considering manufacturing constraints
      - Review existing models in models/ folder
      - Consider assembly, thermal, and structural requirements      
    voting: false

  pcb_layout:
    goal: Plan PCB component placement
    artifacts:
      formats: [svg, plantuml]
      output_dir: pcb/
    instructions: |
      - Create component layout diagrams
      - Consider signal routing and thermal zones      
    voting: false

  final_review:
    goal: Approve the complete design
    artifacts:
      formats: [openscad, svg, plantuml]
      output_dir: final/
    voting: true
    threshold_ready: 0.67

Example: UI Design Template

name: ui_design
description: User interface design workflow

phases:
  wireframe:
    goal: Sketch rough layouts
    artifacts:
      formats: [svg, ascii]
      output_dir: wireframes/
    instructions: |
      - Create rough wireframes - boxes and labels only
      - Focus on layout and flow, not aesthetics
      - One SVG per screen/view      
    voting: false

  mockup:
    goal: Detailed visual mockups
    artifacts:
      formats: [svg]
      output_dir: mockups/
    instructions: |
      - Build on approved wireframes
      - Add visual hierarchy and styling
      - Consider responsive breakpoints      
    voting: true
    threshold_ready: 0.5