GhostQA

AI-powered visual GUI testing via natural language

Overview

GhostQA enables testing of desktop GUI applications (PyQt, GTK, etc.) using AI vision agents. Instead of writing brittle UI test scripts, describe expected behavior in natural language and let AI verify it visually.

The system works by running target applications inside Docker containers with headless display (Xvfb) and noVNC for remote access. AI agents connect to the exposed URL and interact with the application based on natural language test specifications.

Features

• Natural Language Tests - Write test specs in YAML/Markdown describing expected behavior
• AI Vision Agents - Support for ChatGPT Agent Mode, Claude Computer Use, and browser-use
• Docker Isolation - Apps run in containers with Xvfb + noVNC for headless GUI testing
• Screenshot Capture - Automatic screenshot collection during test execution
• Multiple Agent Support - Flexible architecture supporting various AI backends
• CLI Interface - Simple commands to run apps, execute tests, and build images

Quick Start

Installation

# Clone and install
git clone https://gitea.brrd.tech/rob/ghostqa.git
cd ghostqa
pip install -e ".[dev]"

# Build the base Docker image
ghostqa build

# Run an app in test mode
ghostqa run --app development-hub --port 6080

# Execute tests
ghostqa test specs/dashboard.yaml --agent claude --url http://localhost:6080

Example Test Spec

name: Dashboard displays project list
app: development-hub
steps:
  - action: wait_for_window
    timeout: 10s
  - action: verify
    prompt: "Is there a list of projects visible on the left side?"
    expected: true
  - action: click
    prompt: "Click on the first project in the list"
  - action: verify
    prompt: "Does the right panel show a dashboard with todos and goals?"
    expected: true

Architecture

┌─────────────────────────────────────┐
│ Docker Container                    │
│ ┌─────────────┐   ┌──────────────┐ │
│ │ Target App  │──▶│ Xvfb + noVNC │──┼──▶ Public URL (Cloudflare)
│ └─────────────┘   └──────────────┘ │
└─────────────────────────────────────┘
                                       │
                          ┌────────────▼────────────┐
                          │ AI Agent                │
                          │ (ChatGPT, Claude, etc.) │
                          │                         │
                          │ "Click Projects,        │
                          │  verify 5 items shown"  │
                          └─────────────────────────┘

Core Components

Component	Purpose
docker.runner	Build and manage Docker containers with noVNC
docker.base	Base image builder with Xvfb, x11vnc, noVNC
agents.base	Abstract interface for AI vision agents
specs.parser	Parse natural language test specifications
specs.models	Test spec and step data models
runner	Execute tests, coordinate agents and containers

Related Projects

This project is part of the development ecosystem:

• CmdForge - AI-powered CLI tool builder
• CascadingDev - Cascading Development Framework
• Orchestrated Discussions - AI Discussion Framework
• Artifact Editor - Code Artifact Editor
• Ramble - Voice Note Transcription