Commit Graph

9 Commits

Author SHA1 Message Date
rob ab217e247f Allow omitting limit to return all candles in range
Change default limit from 100 to None. When limit is omitted,
EDM returns all candles in the requested range without truncation.

Also increased max limit from 1000 to 10000 for large requests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-08 01:26:31 -04:00
rob e14a607bfc Fix SINCE mode requests returning wrong data
When requesting candles with start date but no end date (SINCE mode),
EDM was skipping gap processing entirely because gap_end was None.
This caused requests to return cached data clipped to the most recent
N candles, ignoring the requested start date.

Fix: Normalize gaps with end=None to use current time, so SINCE mode
requests properly fetch data from the requested start time to now.

Bug symptoms:
- Request start=2026-03-01, limit=100
- Got 100 candles starting from 2026-03-04 (most recent 100)
- Expected 100 candles starting from 2026-03-01

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-08 01:15:47 -04:00
rob c4bcc19241 Fix SQLite database locking under concurrent access
Add busy_timeout (5 seconds) to both async and sync database connections.
This prevents "database is locked" errors when multiple requests try to
access the database simultaneously.

Changes:
- AsyncSQLitePool: Add busy_timeout=5000 and synchronous=NORMAL pragmas
- DatabaseCache: Add WAL mode, busy_timeout, synchronous=NORMAL, and
  connection timeout for the sync database module

The busy_timeout tells SQLite to wait instead of immediately failing
when the database is locked by another connection.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 13:40:58 -04:00
rob bf95490673 Update docs, enable exchanges, and fix tests
- Update CLAUDE.md to reflect actual module structure (add websocket,
  connection_pool, monitoring modules; remove outdated "Planned" note)
- Enable additional exchanges in config.yaml (kucoin, kraken, okx, etc.)
- Fix cache manager to refetch when memory has fewer candles than limit
- Fix benchmark tests asyncio event loop issues (use new_event_loop)
- Update unit tests to match new cache behavior and config

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 13:36:38 -04:00
rob cdc04d010a Add real exchange integration tests for three-tier cache system
Comprehensive tests that fetch actual data from Binance to verify:
- Three-tier caching (memory → database → exchange)
- Gap detection and filling (start, end, middle, multiple gaps)
- Data integrity verification against direct exchange fetch
- Freshness checking for stale data
- Different timeframe handling
- Edge cases (single candle, large ranges, overlapping requests)

Tests are marked with @real_exchange and can be skipped in CI with:
  pytest -m "not real_exchange"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-03 17:37:04 -04:00
rob 958168b3c9 Fix stale data served for limit-only requests
Add freshness check to _detect_gaps() in all cache layers (memory,
database, async_database). For limit-only requests (no start/end),
verify the most recent candle is within 2 intervals of current time.
If stale, flag as a gap so fresh data is fetched from exchange.

This fixes the issue where EDM served hours-old cached data instead
of fetching current candles when clients requested "latest N candles".

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-03 14:34:31 -04:00
rob d2cd47ea95 Add session-scoped connectors and request metrics tracking
- CacheManager.get_candles_with_source() supports per-request connector
  overrides and reports data source (memory/database/exchange)
- AsyncDatabaseCache now receives pool_size/max_overflow from config
- CacheManager.close() properly shuts down async DB connection pool
- /candles endpoint accepts optional session_id for authenticated access
- /candles records metrics on success and errors (latency, cache source)
- Added tests for connector override, source reporting, and metrics

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-03 14:02:56 -04:00
rob e501d61445 Implement M1, M2, M3 milestones - full exchange data manager
M1: Core Infrastructure
- Three-tier caching (memory → database → exchange)
- SQLite async database with connection pooling
- Memory cache with LRU eviction and TTL
- Gap detection and forward-fill gap filling
- Completeness validation for data requests

M2: Multi-Exchange Support
- Generic CCXT connector supporting all ccxt exchanges
- Session-based credential management
- WebSocket streaming with pub/sub subscriptions
- Rate limiting via CCXT's built-in limiter

M3: Production Ready
- Health checks (/health, /health/detailed) and metrics (/metrics)
- Connection pooling with WAL mode for SQLite
- Performance benchmarks (pytest-benchmark)
- BrighterTrading integration tests
- Docker deployment (Dockerfile, docker-compose.yml)
- Documentation (API reference, deployment, integration guide)

Includes 284 tests covering all components.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-03 04:05:30 -04:00
rob 93b5a2299e Initial project setup
Created by development-hub/new-project script
2026-03-02 22:45:06 -04:00