Compare commits
No commits in common. "pages" and "master" have entirely different histories.
|
|
@ -0,0 +1,53 @@
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
*$py.class
|
||||||
|
*.so
|
||||||
|
.Python
|
||||||
|
build/
|
||||||
|
develop-eggs/
|
||||||
|
dist/
|
||||||
|
downloads/
|
||||||
|
eggs/
|
||||||
|
.eggs/
|
||||||
|
lib/
|
||||||
|
lib64/
|
||||||
|
parts/
|
||||||
|
sdist/
|
||||||
|
var/
|
||||||
|
wheels/
|
||||||
|
*.egg-info/
|
||||||
|
.installed.cfg
|
||||||
|
*.egg
|
||||||
|
|
||||||
|
# Virtual environments
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
ENV/
|
||||||
|
|
||||||
|
# IDE
|
||||||
|
.idea/
|
||||||
|
.vscode/
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
|
||||||
|
# OS
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Testing
|
||||||
|
.pytest_cache/
|
||||||
|
.coverage
|
||||||
|
htmlcov/
|
||||||
|
.tox/
|
||||||
|
.nox/
|
||||||
|
|
||||||
|
# mypy
|
||||||
|
.mypy_cache/
|
||||||
|
|
||||||
|
# Local config
|
||||||
|
.env
|
||||||
|
*.local.yaml
|
||||||
|
|
||||||
|
# Documentation symlink (points to project-docs)
|
||||||
|
docs
|
||||||
|
|
@ -0,0 +1,126 @@
|
||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
**GhostQA** - AI-powered visual GUI testing via natural language
|
||||||
|
|
||||||
|
GhostQA enables testing of desktop GUI applications (PyQt, GTK, etc.) using AI vision agents. Instead of writing brittle UI test scripts, describe expected behavior in natural language and let AI verify it visually.
|
||||||
|
|
||||||
|
### How It Works
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────┐
|
||||||
|
│ Docker Container │
|
||||||
|
│ ┌─────────────┐ ┌──────────────┐ │
|
||||||
|
│ │ Target App │──▶│ Xvfb + noVNC │──┼──▶ Public URL (Cloudflare)
|
||||||
|
│ └─────────────┘ └──────────────┘ │
|
||||||
|
└─────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
┌────────────▼────────────┐
|
||||||
|
│ AI Agent │
|
||||||
|
│ (ChatGPT, Claude, etc.) │
|
||||||
|
│ │
|
||||||
|
│ "Click Projects, │
|
||||||
|
│ verify 5 items shown" │
|
||||||
|
└─────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Core Components
|
||||||
|
|
||||||
|
1. **Docker Base Image** - Pre-configured with Xvfb, x11vnc, noVNC for headless GUI
|
||||||
|
2. **Test Runner** - Orchestrates containers and AI agent interactions
|
||||||
|
3. **Test Specs** - YAML/Markdown files describing tests in natural language
|
||||||
|
4. **Results Reporter** - Captures screenshots, AI observations, pass/fail status
|
||||||
|
|
||||||
|
### Example Test Spec
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: Dashboard displays project list
|
||||||
|
app: development-hub
|
||||||
|
steps:
|
||||||
|
- action: wait_for_window
|
||||||
|
timeout: 10s
|
||||||
|
- action: verify
|
||||||
|
prompt: "Is there a list of projects visible on the left side?"
|
||||||
|
expected: true
|
||||||
|
- action: click
|
||||||
|
prompt: "Click on the first project in the list"
|
||||||
|
- action: verify
|
||||||
|
prompt: "Does the right panel show a dashboard with todos and goals?"
|
||||||
|
expected: true
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install for development
|
||||||
|
pip install -e ".[dev]"
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
pytest
|
||||||
|
|
||||||
|
# Build the base Docker image
|
||||||
|
docker build -t ghostqa-base -f docker/Dockerfile.base .
|
||||||
|
|
||||||
|
# Run an app in test mode
|
||||||
|
ghostqa run --app development-hub --expose 6080
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
src/ghostqa/
|
||||||
|
├── __init__.py
|
||||||
|
├── __main__.py # CLI entry point
|
||||||
|
├── docker/
|
||||||
|
│ ├── base.py # Base image builder
|
||||||
|
│ └── runner.py # Container orchestration
|
||||||
|
├── agents/
|
||||||
|
│ ├── base.py # Abstract AI agent interface
|
||||||
|
│ ├── chatgpt.py # ChatGPT agent mode (via browser automation)
|
||||||
|
│ └── claude.py # Claude computer-use API
|
||||||
|
├── specs/
|
||||||
|
│ ├── parser.py # Parse test spec YAML/MD
|
||||||
|
│ └── models.py # Test spec data models
|
||||||
|
├── runner.py # Test execution engine
|
||||||
|
└── reporter.py # Results and screenshots
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Modules
|
||||||
|
|
||||||
|
| Module | Purpose |
|
||||||
|
|--------|---------|
|
||||||
|
| `docker.runner` | Build and manage Docker containers with noVNC |
|
||||||
|
| `agents.base` | Abstract interface for AI vision agents |
|
||||||
|
| `specs.parser` | Parse natural language test specifications |
|
||||||
|
| `runner` | Execute tests, coordinate agents and containers |
|
||||||
|
| `reporter` | Generate test reports with screenshots |
|
||||||
|
|
||||||
|
### Key Paths
|
||||||
|
|
||||||
|
- **Source code**: `src/ghostqa/`
|
||||||
|
- **Tests**: `tests/`
|
||||||
|
- **Docker files**: `docker/`
|
||||||
|
- **Example specs**: `examples/`
|
||||||
|
- **Documentation**: `docs/` (symlink to project-docs)
|
||||||
|
|
||||||
|
## Supported AI Agents
|
||||||
|
|
||||||
|
| Agent | Method | Notes |
|
||||||
|
|-------|--------|-------|
|
||||||
|
| ChatGPT Agent Mode | Browser automation to chat.openai.com | Included in Plus/Pro subscription |
|
||||||
|
| Claude Computer Use | API with vision + actions | Per-token pricing |
|
||||||
|
| Open source (browser-use) | Local LLM or API | Flexible, self-hosted |
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
Documentation lives in `docs/` (symlink to centralized docs system).
|
||||||
|
|
||||||
|
**Before updating docs, read `docs/updating-documentation.md`** for full details on visibility rules and procedures.
|
||||||
|
|
||||||
|
Quick reference:
|
||||||
|
- Edit files in `docs/` folder
|
||||||
|
- Use `public: true` frontmatter for public-facing docs
|
||||||
|
- Deploy: `~/PycharmProjects/project-docs/scripts/build-public-docs.sh ghostqa --deploy`
|
||||||
|
|
@ -0,0 +1,39 @@
|
||||||
|
# GhostQA
|
||||||
|
|
||||||
|
AI-powered visual GUI testing via natural language
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -e .
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
*TODO: Add usage instructions*
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
Full documentation is available at: https://pages.brrd.tech/rob/ghostqa/
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Clone the repository
|
||||||
|
git clone https://gitea.brrd.tech/rob/ghostqa.git
|
||||||
|
cd ghostqa
|
||||||
|
|
||||||
|
# Create virtual environment
|
||||||
|
python -m venv .venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
|
||||||
|
# Install for development
|
||||||
|
pip install -e ".[dev]"
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
*TODO: Add license*
|
||||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -1 +0,0 @@
|
||||||
"use strict";(globalThis.webpackChunkproject_public_docs=globalThis.webpackChunkproject_public_docs||[]).push([[237],{2237(e,t,i){i.r(t),i.d(t,{default:()=>l});i(6540);var o=i(1312),n=i(5500),s=i(1656),r=i(3363),a=i(4848);function l(){const e=(0,o.T)({id:"theme.NotFound.title",message:"Page Not Found"});return(0,a.jsxs)(a.Fragment,{children:[(0,a.jsx)(n.be,{title:e}),(0,a.jsx)(s.A,{children:(0,a.jsx)(r.A,{})})]})}},3363(e,t,i){i.d(t,{A:()=>a});i(6540);var o=i(4164),n=i(1312),s=i(1107),r=i(4848);function a({className:e}){return(0,r.jsx)("main",{className:(0,o.A)("container margin-vert--xl",e),children:(0,r.jsx)("div",{className:"row",children:(0,r.jsxs)("div",{className:"col col--6 col--offset-3",children:[(0,r.jsx)(s.A,{as:"h1",className:"hero__title",children:(0,r.jsx)(n.A,{id:"theme.NotFound.title",description:"The title of the 404 page",children:"Page Not Found"})}),(0,r.jsx)("p",{children:(0,r.jsx)(n.A,{id:"theme.NotFound.p1",description:"The first paragraph of the 404 page",children:"We could not find what you were looking for."})}),(0,r.jsx)("p",{children:(0,r.jsx)(n.A,{id:"theme.NotFound.p2",description:"The 2nd paragraph of the 404 page",children:"Please contact the owner of the site that linked you to the original URL and let them know their link is broken."})})]})})})}}}]);
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
"use strict";(globalThis.webpackChunkproject_public_docs=globalThis.webpackChunkproject_public_docs||[]).push([[221],{4785(e){e.exports=JSON.parse('{"version":{"pluginId":"default","version":"current","label":"Next","banner":null,"badge":false,"noIndex":false,"className":"docs-version-current","isLast":true,"docsSidebars":{"docs":[{"type":"link","href":"/rob/ghostqa/","label":"GhostQA","docId":"overview","unlisted":false}]},"docs":{"overview":{"id":"overview","title":"GhostQA","description":"AI-powered visual GUI testing via natural language","sidebar":"docs"}}}}')}}]);
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
"use strict";(globalThis.webpackChunkproject_public_docs=globalThis.webpackChunkproject_public_docs||[]).push([[647],{7121(e,c,s){s.r(c),s.d(c,{default:()=>t});s(6540);var r=s(4164),u=s(7559),a=s(5500),l=s(2831),o=s(1656),p=s(4848);function t(e){return(0,p.jsx)(a.e3,{className:(0,r.A)(u.G.wrapper.docsPages),children:(0,p.jsx)(o.A,{children:(0,l.v)(e.route.routes)})})}}}]);
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
"use strict";(globalThis.webpackChunkproject_public_docs=globalThis.webpackChunkproject_public_docs||[]).push([[98],{1723(n,e,s){s.r(e),s.d(e,{default:()=>d});s(6540);var r=s(5500);function o(n,e){return`docs-${n}-${e}`}var c=s(3025),t=s(2831),i=s(1463),u=s(4848);function l(n){const{version:e}=n;return(0,u.jsxs)(u.Fragment,{children:[(0,u.jsx)(i.A,{version:e.version,tag:o(e.pluginId,e.version)}),(0,u.jsx)(r.be,{children:e.noIndex&&(0,u.jsx)("meta",{name:"robots",content:"noindex, nofollow"})})]})}function a(n){const{version:e,route:s}=n;return(0,u.jsx)(r.e3,{className:e.className,children:(0,u.jsx)(c.n,{version:e,children:(0,t.v)(s.routes)})})}function d(n){return(0,u.jsxs)(u.Fragment,{children:[(0,u.jsx)(l,{...n}),(0,u.jsx)(a,{...n})]})}}}]);
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -1 +0,0 @@
|
||||||
"use strict";(globalThis.webpackChunkproject_public_docs=globalThis.webpackChunkproject_public_docs||[]).push([[742],{7093(c){c.exports=JSON.parse('{"name":"docusaurus-plugin-content-docs","id":"default"}')}}]);
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -1,61 +0,0 @@
|
||||||
/* NProgress, (c) 2013, 2014 Rico Sta. Cruz - http://ricostacruz.com/nprogress
|
|
||||||
* @license MIT */
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @license React
|
|
||||||
* react-dom-client.production.js
|
|
||||||
*
|
|
||||||
* Copyright (c) Meta Platforms, Inc. and affiliates.
|
|
||||||
*
|
|
||||||
* This source code is licensed under the MIT license found in the
|
|
||||||
* LICENSE file in the root directory of this source tree.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @license React
|
|
||||||
* react-dom.production.js
|
|
||||||
*
|
|
||||||
* Copyright (c) Meta Platforms, Inc. and affiliates.
|
|
||||||
*
|
|
||||||
* This source code is licensed under the MIT license found in the
|
|
||||||
* LICENSE file in the root directory of this source tree.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @license React
|
|
||||||
* react-jsx-runtime.production.js
|
|
||||||
*
|
|
||||||
* Copyright (c) Meta Platforms, Inc. and affiliates.
|
|
||||||
*
|
|
||||||
* This source code is licensed under the MIT license found in the
|
|
||||||
* LICENSE file in the root directory of this source tree.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @license React
|
|
||||||
* react.production.js
|
|
||||||
*
|
|
||||||
* Copyright (c) Meta Platforms, Inc. and affiliates.
|
|
||||||
*
|
|
||||||
* This source code is licensed under the MIT license found in the
|
|
||||||
* LICENSE file in the root directory of this source tree.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @license React
|
|
||||||
* scheduler.production.js
|
|
||||||
*
|
|
||||||
* Copyright (c) Meta Platforms, Inc. and affiliates.
|
|
||||||
*
|
|
||||||
* This source code is licensed under the MIT license found in the
|
|
||||||
* LICENSE file in the root directory of this source tree.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/** @license React v16.13.1
|
|
||||||
* react-is.production.min.js
|
|
||||||
*
|
|
||||||
* Copyright (c) Facebook, Inc. and its affiliates.
|
|
||||||
*
|
|
||||||
* This source code is licensed under the MIT license found in the
|
|
||||||
* LICENSE file in the root directory of this source tree.
|
|
||||||
*/
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
(()=>{"use strict";var e,r,t,a,o,n={},i={};function l(e){var r=i[e];if(void 0!==r)return r.exports;var t=i[e]={id:e,loaded:!1,exports:{}};return n[e].call(t.exports,t,t.exports,l),t.loaded=!0,t.exports}l.m=n,l.c=i,e=[],l.O=(r,t,a,o)=>{if(!t){var n=1/0;for(d=0;d<e.length;d++){for(var[t,a,o]=e[d],i=!0,c=0;c<t.length;c++)(!1&o||n>=o)&&Object.keys(l.O).every(e=>l.O[e](t[c]))?t.splice(c--,1):(i=!1,o<n&&(n=o));if(i){e.splice(d--,1);var u=a();void 0!==u&&(r=u)}}return r}o=o||0;for(var d=e.length;d>0&&e[d-1][2]>o;d--)e[d]=e[d-1];e[d]=[t,a,o]},l.n=e=>{var r=e&&e.__esModule?()=>e.default:()=>e;return l.d(r,{a:r}),r},t=Object.getPrototypeOf?e=>Object.getPrototypeOf(e):e=>e.__proto__,l.t=function(e,a){if(1&a&&(e=this(e)),8&a)return e;if("object"==typeof e&&e){if(4&a&&e.__esModule)return e;if(16&a&&"function"==typeof e.then)return e}var o=Object.create(null);l.r(o);var n={};r=r||[null,t({}),t([]),t(t)];for(var i=2&a&&e;("object"==typeof i||"function"==typeof i)&&!~r.indexOf(i);i=t(i))Object.getOwnPropertyNames(i).forEach(r=>n[r]=()=>e[r]);return n.default=()=>e,l.d(o,n),o},l.d=(e,r)=>{for(var t in r)l.o(r,t)&&!l.o(e,t)&&Object.defineProperty(e,t,{enumerable:!0,get:r[t]})},l.f={},l.e=e=>Promise.all(Object.keys(l.f).reduce((r,t)=>(l.f[t](e,r),r),[])),l.u=e=>"assets/js/"+({48:"a94703ab",98:"a7bd4aaa",221:"44a29501",401:"17896441",413:"1db64337",647:"5e95c892",742:"aba21aa0"}[e]||e)+"."+{48:"b8c77466",98:"3ba34601",221:"a20a9f72",237:"447ba118",401:"a2525508",413:"3740f1b5",647:"a3b66919",742:"4a552a5c"}[e]+".js",l.miniCssF=e=>{},l.o=(e,r)=>Object.prototype.hasOwnProperty.call(e,r),a={},o="project-public-docs:",l.l=(e,r,t,n)=>{if(a[e])a[e].push(r);else{var i,c;if(void 0!==t)for(var u=document.getElementsByTagName("script"),d=0;d<u.length;d++){var s=u[d];if(s.getAttribute("src")==e||s.getAttribute("data-webpack")==o+t){i=s;break}}i||(c=!0,(i=document.createElement("script")).charset="utf-8",l.nc&&i.setAttribute("nonce",l.nc),i.setAttribute("data-webpack",o+t),i.src=e),a[e]=[r];var f=(r,t)=>{i.onerror=i.onload=null,clearTimeout(p);var o=a[e];if(delete a[e],i.parentNode&&i.parentNode.removeChild(i),o&&o.forEach(e=>e(t)),r)return r(t)},p=setTimeout(f.bind(null,void 0,{type:"timeout",target:i}),12e4);i.onerror=f.bind(null,i.onerror),i.onload=f.bind(null,i.onload),c&&document.head.appendChild(i)}},l.r=e=>{"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},l.p="/rob/ghostqa/",l.gca=function(e){return e={17896441:"401",a94703ab:"48",a7bd4aaa:"98","44a29501":"221","1db64337":"413","5e95c892":"647",aba21aa0:"742"}[e]||e,l.p+l.u(e)},(()=>{var e={354:0,869:0};l.f.j=(r,t)=>{var a=l.o(e,r)?e[r]:void 0;if(0!==a)if(a)t.push(a[2]);else if(/^(354|869)$/.test(r))e[r]=0;else{var o=new Promise((t,o)=>a=e[r]=[t,o]);t.push(a[2]=o);var n=l.p+l.u(r),i=new Error;l.l(n,t=>{if(l.o(e,r)&&(0!==(a=e[r])&&(e[r]=void 0),a)){var o=t&&("load"===t.type?"missing":t.type),n=t&&t.target&&t.target.src;i.message="Loading chunk "+r+" failed.\n("+o+": "+n+")",i.name="ChunkLoadError",i.type=o,i.request=n,a[1](i)}},"chunk-"+r,r)}},l.O.j=r=>0===e[r];var r=(r,t)=>{var a,o,[n,i,c]=t,u=0;if(n.some(r=>0!==e[r])){for(a in i)l.o(i,a)&&(l.m[a]=i[a]);if(c)var d=c(l)}for(r&&r(t);u<n.length;u++)o=n[u],l.o(e,o)&&e[o]&&e[o][0](),e[o]=0;return l.O(d)},t=globalThis.webpackChunkproject_public_docs=globalThis.webpackChunkproject_public_docs||[];t.forEach(r.bind(null,0)),t.push=r.bind(null,t.push.bind(t))})()})();
|
|
||||||
|
|
@ -0,0 +1,62 @@
|
||||||
|
# GhostQA Base Image
|
||||||
|
# Pre-configured headless display with noVNC for AI visual testing
|
||||||
|
|
||||||
|
FROM python:3.11-slim
|
||||||
|
|
||||||
|
# Install display and VNC dependencies
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
xvfb \
|
||||||
|
x11vnc \
|
||||||
|
novnc \
|
||||||
|
websockify \
|
||||||
|
# Qt/GUI dependencies
|
||||||
|
libxcb-cursor0 \
|
||||||
|
libxcb-icccm4 \
|
||||||
|
libxcb-image0 \
|
||||||
|
libxcb-keysyms1 \
|
||||||
|
libxcb-randr0 \
|
||||||
|
libxcb-render-util0 \
|
||||||
|
libxcb-shape0 \
|
||||||
|
libxcb-xfixes0 \
|
||||||
|
libxcb-xinerama0 \
|
||||||
|
libxcb-xkb1 \
|
||||||
|
libxkbcommon-x11-0 \
|
||||||
|
libegl1 \
|
||||||
|
libgl1 \
|
||||||
|
libfontconfig1 \
|
||||||
|
libdbus-1-3 \
|
||||||
|
# Utilities
|
||||||
|
procps \
|
||||||
|
net-tools \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Set up virtual display
|
||||||
|
ENV DISPLAY=:99
|
||||||
|
ENV QT_QPA_PLATFORM=xcb
|
||||||
|
|
||||||
|
# Create startup script
|
||||||
|
RUN echo '#!/bin/bash\n\
|
||||||
|
set -e\n\
|
||||||
|
\n\
|
||||||
|
# Start virtual framebuffer\n\
|
||||||
|
Xvfb :99 -screen 0 ${SCREEN_WIDTH:-1280}x${SCREEN_HEIGHT:-720}x24 &\n\
|
||||||
|
sleep 1\n\
|
||||||
|
\n\
|
||||||
|
# Start VNC server\n\
|
||||||
|
x11vnc -display :99 -forever -shared -rfbport 5900 -nopw &\n\
|
||||||
|
sleep 1\n\
|
||||||
|
\n\
|
||||||
|
# Start noVNC (web-based VNC client)\n\
|
||||||
|
websockify --web=/usr/share/novnc/ ${NOVNC_PORT:-6080} localhost:5900 &\n\
|
||||||
|
\n\
|
||||||
|
echo "GhostQA display ready on port ${NOVNC_PORT:-6080}"\n\
|
||||||
|
echo "Connect via browser: http://localhost:${NOVNC_PORT:-6080}/vnc.html"\n\
|
||||||
|
\n\
|
||||||
|
# Run the target application\n\
|
||||||
|
exec "$@"\n\
|
||||||
|
' > /usr/local/bin/ghostqa-start && chmod +x /usr/local/bin/ghostqa-start
|
||||||
|
|
||||||
|
EXPOSE 6080
|
||||||
|
|
||||||
|
ENTRYPOINT ["/usr/local/bin/ghostqa-start"]
|
||||||
|
CMD ["bash"]
|
||||||
|
|
@ -0,0 +1,36 @@
|
||||||
|
# Example GhostQA test spec for Development Hub
|
||||||
|
name: Development Hub Dashboard
|
||||||
|
app: development-hub
|
||||||
|
description: Verify the Development Hub GUI loads and displays projects correctly
|
||||||
|
tags: [smoke, gui]
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Wait for app window
|
||||||
|
action: wait_for_window
|
||||||
|
timeout: 15s
|
||||||
|
|
||||||
|
- name: Verify project list visible
|
||||||
|
action: verify
|
||||||
|
prompt: "Is there a list of projects visible on the left side of the window?"
|
||||||
|
expected: true
|
||||||
|
|
||||||
|
- name: Count projects
|
||||||
|
action: verify
|
||||||
|
prompt: "How many projects are shown in the list? Just give me the number."
|
||||||
|
|
||||||
|
- name: Click first project
|
||||||
|
action: click
|
||||||
|
prompt: "Click on the first project in the list on the left side"
|
||||||
|
|
||||||
|
- name: Verify dashboard loaded
|
||||||
|
action: verify
|
||||||
|
prompt: "Does the right panel now show a dashboard with sections like 'GOALS', 'MILESTONES', or 'TODOs'?"
|
||||||
|
expected: true
|
||||||
|
|
||||||
|
- name: Check for todos section
|
||||||
|
action: verify
|
||||||
|
prompt: "Is there a 'TODOs' section visible with priority levels like 'HIGH PRIORITY'?"
|
||||||
|
expected: true
|
||||||
|
|
||||||
|
- name: Take final screenshot
|
||||||
|
action: screenshot
|
||||||
47
index.html
47
index.html
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,35 @@
|
||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=61.0", "wheel"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "ghostqa"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "AI-powered visual GUI testing via natural language"
|
||||||
|
readme = "README.md"
|
||||||
|
requires-python = ">=3.10"
|
||||||
|
dependencies = [
|
||||||
|
"pyyaml>=6.0",
|
||||||
|
"httpx>=0.25",
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.optional-dependencies]
|
||||||
|
dev = [
|
||||||
|
"pytest>=7.0",
|
||||||
|
"pytest-cov>=4.0",
|
||||||
|
]
|
||||||
|
claude = [
|
||||||
|
"anthropic>=0.18",
|
||||||
|
]
|
||||||
|
browser-use = [
|
||||||
|
"browser-use>=0.1",
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.scripts]
|
||||||
|
ghostqa = "ghostqa.__main__:main"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
where = ["src"]
|
||||||
|
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
testpaths = ["tests"]
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
<?xml version="1.0" encoding="UTF-8"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1"><url><loc>https://pages.brrd.tech/rob/ghostqa/</loc><changefreq>weekly</changefreq><priority>0.5</priority></url></urlset>
|
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
"""GhostQA - AI-powered visual GUI testing via natural language."""
|
||||||
|
|
||||||
|
__version__ = "0.1.0"
|
||||||
|
|
@ -0,0 +1,51 @@
|
||||||
|
"""CLI entry point for GhostQA."""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""Main CLI entry point."""
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
prog="ghostqa",
|
||||||
|
description="AI-powered visual GUI testing via natural language",
|
||||||
|
)
|
||||||
|
subparsers = parser.add_subparsers(dest="command", help="Commands")
|
||||||
|
|
||||||
|
# ghostqa run - Run an app in test mode
|
||||||
|
run_parser = subparsers.add_parser("run", help="Run an app in test container")
|
||||||
|
run_parser.add_argument("--app", required=True, help="App to run (name or path)")
|
||||||
|
run_parser.add_argument("--port", type=int, default=6080, help="noVNC port (default: 6080)")
|
||||||
|
run_parser.add_argument("--build", action="store_true", help="Rebuild container image")
|
||||||
|
|
||||||
|
# ghostqa test - Run test specs
|
||||||
|
test_parser = subparsers.add_parser("test", help="Run test specifications")
|
||||||
|
test_parser.add_argument("spec", nargs="?", help="Test spec file (default: all in specs/)")
|
||||||
|
test_parser.add_argument("--agent", choices=["chatgpt", "claude", "browser-use"], default="claude")
|
||||||
|
test_parser.add_argument("--url", help="URL where app is exposed")
|
||||||
|
|
||||||
|
# ghostqa build - Build base image
|
||||||
|
build_parser = subparsers.add_parser("build", help="Build GhostQA base Docker image")
|
||||||
|
build_parser.add_argument("--no-cache", action="store_true", help="Build without cache")
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if args.command is None:
|
||||||
|
parser.print_help()
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
if args.command == "run":
|
||||||
|
from ghostqa.docker.runner import run_app
|
||||||
|
run_app(args.app, args.port, rebuild=args.build)
|
||||||
|
|
||||||
|
elif args.command == "test":
|
||||||
|
from ghostqa.runner import run_tests
|
||||||
|
run_tests(args.spec, agent=args.agent, url=args.url)
|
||||||
|
|
||||||
|
elif args.command == "build":
|
||||||
|
from ghostqa.docker.base import build_base_image
|
||||||
|
build_base_image(no_cache=args.no_cache)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
|
@ -0,0 +1,5 @@
|
||||||
|
"""AI agents for visual GUI testing."""
|
||||||
|
|
||||||
|
from ghostqa.agents.base import Agent
|
||||||
|
|
||||||
|
__all__ = ["Agent"]
|
||||||
|
|
@ -0,0 +1,100 @@
|
||||||
|
"""Base agent interface for AI visual testing."""
|
||||||
|
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ActionResult:
|
||||||
|
"""Result of an agent action."""
|
||||||
|
|
||||||
|
success: bool
|
||||||
|
message: str
|
||||||
|
screenshot: bytes | None = None
|
||||||
|
observation: str = ""
|
||||||
|
raw_response: Any = None
|
||||||
|
|
||||||
|
|
||||||
|
class Agent(ABC):
|
||||||
|
"""Abstract base class for AI agents that can see and interact with GUIs."""
|
||||||
|
|
||||||
|
def __init__(self, url: str):
|
||||||
|
"""Initialize agent with target URL.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
url: URL where the app's noVNC is exposed
|
||||||
|
"""
|
||||||
|
self.url = url
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def connect(self) -> bool:
|
||||||
|
"""Connect to the target application.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
True if connection successful
|
||||||
|
"""
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def screenshot(self) -> bytes:
|
||||||
|
"""Take a screenshot of the current state.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
PNG image bytes
|
||||||
|
"""
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def verify(self, prompt: str) -> ActionResult:
|
||||||
|
"""Verify something about the current screen.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
prompt: Natural language description of what to verify
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
ActionResult with observation
|
||||||
|
"""
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def click(self, prompt: str) -> ActionResult:
|
||||||
|
"""Click on an element described in natural language.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
prompt: Description of what to click (e.g., "the Save button")
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
ActionResult indicating success/failure
|
||||||
|
"""
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def type_text(self, text: str, prompt: str | None = None) -> ActionResult:
|
||||||
|
"""Type text, optionally into a specific field.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Text to type
|
||||||
|
prompt: Optional description of where to type
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
ActionResult indicating success/failure
|
||||||
|
"""
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def wait_for(self, prompt: str, timeout: float = 10.0) -> ActionResult:
|
||||||
|
"""Wait for a condition to be true.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
prompt: Description of what to wait for
|
||||||
|
timeout: Maximum time to wait in seconds
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
ActionResult indicating if condition was met
|
||||||
|
"""
|
||||||
|
pass
|
||||||
|
|
||||||
|
def disconnect(self):
|
||||||
|
"""Disconnect from the target application."""
|
||||||
|
pass
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
"""Docker container management for GhostQA."""
|
||||||
|
|
@ -0,0 +1,32 @@
|
||||||
|
"""Build GhostQA base Docker image."""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def build_base_image(no_cache: bool = False):
|
||||||
|
"""Build the GhostQA base Docker image.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
no_cache: Build without using cache
|
||||||
|
"""
|
||||||
|
dockerfile = Path(__file__).parent.parent.parent.parent / "docker" / "Dockerfile.base"
|
||||||
|
|
||||||
|
if not dockerfile.exists():
|
||||||
|
# Try package location
|
||||||
|
import ghostqa
|
||||||
|
package_dir = Path(ghostqa.__file__).parent
|
||||||
|
dockerfile = package_dir.parent.parent / "docker" / "Dockerfile.base"
|
||||||
|
|
||||||
|
if not dockerfile.exists():
|
||||||
|
print(f"Error: Dockerfile.base not found")
|
||||||
|
return
|
||||||
|
|
||||||
|
cmd = ["docker", "build", "-t", "ghostqa-base", "-f", str(dockerfile), str(dockerfile.parent)]
|
||||||
|
|
||||||
|
if no_cache:
|
||||||
|
cmd.insert(2, "--no-cache")
|
||||||
|
|
||||||
|
print("Building GhostQA base image...")
|
||||||
|
subprocess.run(cmd, check=True)
|
||||||
|
print("Base image built successfully: ghostqa-base")
|
||||||
|
|
@ -0,0 +1,81 @@
|
||||||
|
"""Docker container runner for GUI apps."""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def run_app(app: str, port: int = 6080, rebuild: bool = False):
|
||||||
|
"""Run an application in a GhostQA container.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
app: Application name or path to project
|
||||||
|
port: Port to expose noVNC on
|
||||||
|
rebuild: Whether to rebuild the container image
|
||||||
|
"""
|
||||||
|
# Resolve app path
|
||||||
|
if "/" in app or Path(app).exists():
|
||||||
|
app_path = Path(app).resolve()
|
||||||
|
else:
|
||||||
|
# Assume it's a project in PycharmProjects
|
||||||
|
app_path = Path.home() / "PycharmProjects" / app
|
||||||
|
|
||||||
|
if not app_path.exists():
|
||||||
|
print(f"Error: App not found at {app_path}")
|
||||||
|
return
|
||||||
|
|
||||||
|
app_name = app_path.name.lower().replace("-", "_").replace(" ", "_")
|
||||||
|
image_name = f"ghostqa-{app_name}"
|
||||||
|
|
||||||
|
# Check for Dockerfile in app, otherwise use base
|
||||||
|
dockerfile = app_path / "Dockerfile.ghostqa"
|
||||||
|
if not dockerfile.exists():
|
||||||
|
dockerfile = app_path / "Dockerfile"
|
||||||
|
|
||||||
|
if dockerfile.exists():
|
||||||
|
# Build app-specific image
|
||||||
|
if rebuild or not _image_exists(image_name):
|
||||||
|
print(f"Building image: {image_name}")
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "build", "-t", image_name, "-f", str(dockerfile), str(app_path)],
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
# Use base image with app mounted
|
||||||
|
image_name = "ghostqa-base"
|
||||||
|
if not _image_exists(image_name):
|
||||||
|
print("Base image not found. Building...")
|
||||||
|
from ghostqa.docker.base import build_base_image
|
||||||
|
build_base_image()
|
||||||
|
|
||||||
|
# Run container
|
||||||
|
print(f"Starting {app_name} on port {port}...")
|
||||||
|
print(f"Connect via browser: http://localhost:{port}/vnc.html")
|
||||||
|
|
||||||
|
cmd = [
|
||||||
|
"docker", "run", "-it", "--rm",
|
||||||
|
"-p", f"{port}:6080",
|
||||||
|
"-v", f"{app_path}:/app",
|
||||||
|
"-w", "/app",
|
||||||
|
"-e", f"NOVNC_PORT=6080",
|
||||||
|
image_name,
|
||||||
|
]
|
||||||
|
|
||||||
|
# If using base image, add command to run the app
|
||||||
|
if not (app_path / "Dockerfile.ghostqa").exists() and not (app_path / "Dockerfile").exists():
|
||||||
|
# Try to detect how to run the app
|
||||||
|
if (app_path / "pyproject.toml").exists():
|
||||||
|
cmd.extend(["bash", "-c", "pip install -e . && python -m " + app_name])
|
||||||
|
else:
|
||||||
|
cmd.append("bash")
|
||||||
|
|
||||||
|
subprocess.run(cmd)
|
||||||
|
|
||||||
|
|
||||||
|
def _image_exists(name: str) -> bool:
|
||||||
|
"""Check if a Docker image exists."""
|
||||||
|
result = subprocess.run(
|
||||||
|
["docker", "images", "-q", name],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
return bool(result.stdout.strip())
|
||||||
|
|
@ -0,0 +1,217 @@
|
||||||
|
"""Test execution engine."""
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from ghostqa.specs.models import TestSpec, TestStep, ActionType
|
||||||
|
from ghostqa.specs.parser import parse_spec, find_specs
|
||||||
|
from ghostqa.agents.base import Agent, ActionResult
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class StepResult:
|
||||||
|
"""Result of executing a single test step."""
|
||||||
|
|
||||||
|
step: TestStep
|
||||||
|
passed: bool
|
||||||
|
message: str
|
||||||
|
duration: float
|
||||||
|
screenshot: bytes | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TestResult:
|
||||||
|
"""Result of executing a complete test spec."""
|
||||||
|
|
||||||
|
spec: TestSpec
|
||||||
|
passed: bool
|
||||||
|
step_results: list[StepResult] = field(default_factory=list)
|
||||||
|
started_at: datetime = field(default_factory=datetime.now)
|
||||||
|
duration: float = 0.0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def failed_steps(self) -> list[StepResult]:
|
||||||
|
"""Get list of failed steps."""
|
||||||
|
return [r for r in self.step_results if not r.passed]
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunner:
|
||||||
|
"""Execute test specifications using an AI agent."""
|
||||||
|
|
||||||
|
def __init__(self, agent: Agent):
|
||||||
|
"""Initialize runner with an agent.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
agent: AI agent to use for testing
|
||||||
|
"""
|
||||||
|
self.agent = agent
|
||||||
|
|
||||||
|
def run_spec(self, spec: TestSpec) -> TestResult:
|
||||||
|
"""Run a single test specification.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
spec: Test specification to run
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
TestResult with pass/fail status
|
||||||
|
"""
|
||||||
|
result = TestResult(spec=spec, passed=True)
|
||||||
|
start_time = datetime.now()
|
||||||
|
|
||||||
|
print(f"\nRunning: {spec.name}")
|
||||||
|
print(f" App: {spec.app}")
|
||||||
|
print(f" Steps: {spec.step_count}")
|
||||||
|
print()
|
||||||
|
|
||||||
|
for i, step in enumerate(spec.steps, 1):
|
||||||
|
step_name = step.name or f"Step {i}"
|
||||||
|
print(f" [{i}/{spec.step_count}] {step_name}...", end=" ", flush=True)
|
||||||
|
|
||||||
|
step_start = datetime.now()
|
||||||
|
step_result = self._run_step(step)
|
||||||
|
step_result.duration = (datetime.now() - step_start).total_seconds()
|
||||||
|
|
||||||
|
result.step_results.append(step_result)
|
||||||
|
|
||||||
|
if step_result.passed:
|
||||||
|
print(f"PASS ({step_result.duration:.1f}s)")
|
||||||
|
else:
|
||||||
|
print(f"FAIL ({step_result.duration:.1f}s)")
|
||||||
|
print(f" {step_result.message}")
|
||||||
|
result.passed = False
|
||||||
|
|
||||||
|
result.duration = (datetime.now() - start_time).total_seconds()
|
||||||
|
|
||||||
|
print()
|
||||||
|
if result.passed:
|
||||||
|
print(f" PASSED in {result.duration:.1f}s")
|
||||||
|
else:
|
||||||
|
print(f" FAILED ({len(result.failed_steps)} failures) in {result.duration:.1f}s")
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
def _run_step(self, step: TestStep) -> StepResult:
|
||||||
|
"""Run a single test step.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
step: Step to execute
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
StepResult
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
if step.action == ActionType.WAIT_FOR_WINDOW:
|
||||||
|
action_result = self.agent.wait_for(
|
||||||
|
"The application window is fully loaded and visible",
|
||||||
|
timeout=step.timeout,
|
||||||
|
)
|
||||||
|
|
||||||
|
elif step.action == ActionType.VERIFY:
|
||||||
|
action_result = self.agent.verify(step.prompt)
|
||||||
|
|
||||||
|
# Check expected value if provided
|
||||||
|
if step.expected is not None:
|
||||||
|
if isinstance(step.expected, bool):
|
||||||
|
# Check if response indicates true/false
|
||||||
|
obs_lower = action_result.observation.lower()
|
||||||
|
is_positive = any(
|
||||||
|
word in obs_lower
|
||||||
|
for word in ["yes", "true", "correct", "visible", "shown", "present"]
|
||||||
|
)
|
||||||
|
is_negative = any(
|
||||||
|
word in obs_lower
|
||||||
|
for word in ["no", "false", "not", "cannot", "isn't", "hidden"]
|
||||||
|
)
|
||||||
|
|
||||||
|
if step.expected and not is_positive:
|
||||||
|
action_result.success = False
|
||||||
|
action_result.message = f"Expected true, got: {action_result.observation}"
|
||||||
|
elif not step.expected and not is_negative:
|
||||||
|
action_result.success = False
|
||||||
|
action_result.message = f"Expected false, got: {action_result.observation}"
|
||||||
|
|
||||||
|
elif step.action == ActionType.CLICK:
|
||||||
|
action_result = self.agent.click(step.prompt)
|
||||||
|
|
||||||
|
elif step.action == ActionType.TYPE:
|
||||||
|
action_result = self.agent.type_text(step.text, step.prompt)
|
||||||
|
|
||||||
|
elif step.action == ActionType.WAIT:
|
||||||
|
action_result = self.agent.wait_for(step.prompt, step.timeout)
|
||||||
|
|
||||||
|
elif step.action == ActionType.SCREENSHOT:
|
||||||
|
screenshot = self.agent.screenshot()
|
||||||
|
action_result = ActionResult(
|
||||||
|
success=True,
|
||||||
|
message="Screenshot captured",
|
||||||
|
screenshot=screenshot,
|
||||||
|
)
|
||||||
|
|
||||||
|
else:
|
||||||
|
action_result = ActionResult(
|
||||||
|
success=False,
|
||||||
|
message=f"Unknown action: {step.action}",
|
||||||
|
)
|
||||||
|
|
||||||
|
return StepResult(
|
||||||
|
step=step,
|
||||||
|
passed=action_result.success,
|
||||||
|
message=action_result.message,
|
||||||
|
duration=0.0,
|
||||||
|
screenshot=action_result.screenshot,
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
return StepResult(
|
||||||
|
step=step,
|
||||||
|
passed=False,
|
||||||
|
message=str(e),
|
||||||
|
duration=0.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def run_tests(spec_path: str | None, agent: str = "claude", url: str | None = None):
|
||||||
|
"""Run test specifications.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
spec_path: Path to spec file or directory (None = examples/)
|
||||||
|
agent: Agent type to use
|
||||||
|
url: URL where app is exposed
|
||||||
|
"""
|
||||||
|
# Find specs
|
||||||
|
if spec_path is None:
|
||||||
|
spec_dir = Path(__file__).parent.parent.parent / "examples"
|
||||||
|
specs = find_specs(spec_dir)
|
||||||
|
elif Path(spec_path).is_dir():
|
||||||
|
specs = find_specs(spec_path)
|
||||||
|
else:
|
||||||
|
specs = [Path(spec_path)]
|
||||||
|
|
||||||
|
if not specs:
|
||||||
|
print("No test specs found")
|
||||||
|
return
|
||||||
|
|
||||||
|
print(f"Found {len(specs)} test spec(s)")
|
||||||
|
|
||||||
|
# Create agent
|
||||||
|
if url is None:
|
||||||
|
url = "http://localhost:6080"
|
||||||
|
|
||||||
|
if agent == "claude":
|
||||||
|
# TODO: Implement Claude agent
|
||||||
|
print("Claude agent not yet implemented")
|
||||||
|
return
|
||||||
|
elif agent == "chatgpt":
|
||||||
|
# TODO: Implement ChatGPT agent
|
||||||
|
print("ChatGPT agent not yet implemented")
|
||||||
|
return
|
||||||
|
else:
|
||||||
|
print(f"Unknown agent: {agent}")
|
||||||
|
return
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
# runner = TestRunner(agent_instance)
|
||||||
|
# for spec_file in specs:
|
||||||
|
# spec = parse_spec(spec_file)
|
||||||
|
# result = runner.run_spec(spec)
|
||||||
|
|
@ -0,0 +1,6 @@
|
||||||
|
"""Test specification parsing and models."""
|
||||||
|
|
||||||
|
from ghostqa.specs.models import TestSpec, TestStep
|
||||||
|
from ghostqa.specs.parser import parse_spec
|
||||||
|
|
||||||
|
__all__ = ["TestSpec", "TestStep", "parse_spec"]
|
||||||
|
|
@ -0,0 +1,44 @@
|
||||||
|
"""Test specification data models."""
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
class ActionType(Enum):
|
||||||
|
"""Types of test actions."""
|
||||||
|
|
||||||
|
VERIFY = "verify"
|
||||||
|
CLICK = "click"
|
||||||
|
TYPE = "type"
|
||||||
|
WAIT = "wait"
|
||||||
|
WAIT_FOR_WINDOW = "wait_for_window"
|
||||||
|
SCREENSHOT = "screenshot"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TestStep:
|
||||||
|
"""A single step in a test specification."""
|
||||||
|
|
||||||
|
action: ActionType
|
||||||
|
prompt: str = ""
|
||||||
|
text: str = "" # For type action
|
||||||
|
expected: Any = None # For verify action
|
||||||
|
timeout: float = 10.0
|
||||||
|
name: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TestSpec:
|
||||||
|
"""A complete test specification."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
app: str
|
||||||
|
steps: list[TestStep] = field(default_factory=list)
|
||||||
|
description: str = ""
|
||||||
|
tags: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def step_count(self) -> int:
|
||||||
|
"""Number of steps in this spec."""
|
||||||
|
return len(self.steps)
|
||||||
|
|
@ -0,0 +1,69 @@
|
||||||
|
"""Parse test specification files."""
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
from ghostqa.specs.models import TestSpec, TestStep, ActionType
|
||||||
|
|
||||||
|
|
||||||
|
def parse_spec(path: str | Path) -> TestSpec:
|
||||||
|
"""Parse a test specification file.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
path: Path to YAML spec file
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
TestSpec instance
|
||||||
|
"""
|
||||||
|
path = Path(path)
|
||||||
|
|
||||||
|
with open(path) as f:
|
||||||
|
data = yaml.safe_load(f)
|
||||||
|
|
||||||
|
steps = []
|
||||||
|
for step_data in data.get("steps", []):
|
||||||
|
action_str = step_data.get("action", "verify")
|
||||||
|
action = ActionType(action_str)
|
||||||
|
|
||||||
|
# Parse timeout (handle "10s" format)
|
||||||
|
timeout = step_data.get("timeout", 10.0)
|
||||||
|
if isinstance(timeout, str) and timeout.endswith("s"):
|
||||||
|
timeout = float(timeout[:-1])
|
||||||
|
|
||||||
|
step = TestStep(
|
||||||
|
action=action,
|
||||||
|
prompt=step_data.get("prompt", ""),
|
||||||
|
text=step_data.get("text", ""),
|
||||||
|
expected=step_data.get("expected"),
|
||||||
|
timeout=timeout,
|
||||||
|
name=step_data.get("name", ""),
|
||||||
|
)
|
||||||
|
steps.append(step)
|
||||||
|
|
||||||
|
return TestSpec(
|
||||||
|
name=data.get("name", path.stem),
|
||||||
|
app=data.get("app", ""),
|
||||||
|
steps=steps,
|
||||||
|
description=data.get("description", ""),
|
||||||
|
tags=data.get("tags", []),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def find_specs(directory: str | Path) -> list[Path]:
|
||||||
|
"""Find all test spec files in a directory.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
directory: Directory to search
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of spec file paths
|
||||||
|
"""
|
||||||
|
directory = Path(directory)
|
||||||
|
specs = []
|
||||||
|
|
||||||
|
for pattern in ["*.yaml", "*.yml"]:
|
||||||
|
specs.extend(directory.glob(pattern))
|
||||||
|
specs.extend(directory.glob(f"**/{pattern}"))
|
||||||
|
|
||||||
|
return sorted(set(specs))
|
||||||
Loading…
Reference in New Issue