Commit Graph

85 Commits

Author SHA1 Message Date
rob 307f251576 Add backtest debugging features: timestamps, chart markers, and alerts
- Add Open/Close Time columns to Trades Executed table
- Display trade markers (buy/sell arrows) automatically on chart
- Clear markers when closing results dialog or running new test
- Collect notify_user alerts with timestamps during backtesting
- Display Strategy Alerts section in backtest results
- Fix timestamp conversion: use unit='s' for EDM timestamps (not 'ms')
- Fix trade datetime extraction using data feed's datetime method

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 23:34:55 -04:00
rob 3e6463e4b3 Add Bollinger %B indicator and fix credential handling
- Add BollingerPercentB indicator class in backend (BOL%B)
  - Calculates %B = (Close - Lower) / (Upper - Lower)
  - Shows where price is relative to Bollinger Bands
  - Values: 1.0 = upper band, 0.5 = middle, 0.0 = lower
- Add BollingerPercentB JavaScript class for frontend display
  - Creates its own chart (chart4) like RSI/MACD
  - Includes chart binding for synchronized scrolling
- Add chart4 HTML container and data plumbing
- Fix credential update when reconnecting with changed API keys
- Add TESTNET_MODE config setting for live trading mode control

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 06:31:31 -04:00
rob 21d449d877 Fix KuCoin balance fetching and add balance refresh functionality
- Add KuCoin-specific balance fetching with type='trade' parameter
- Reinitialize ccxt client in refresh_balances() to fix pickle corruption
- Force reconnection when exchange is restored from database cache
- Add balance refresh button and socket handler in frontend
- Fix template null check for balances display
- Clean up DataCache and candles imports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 06:10:31 -04:00
rob 91f51cd71f Fix set_flag blocks being dropped after trade_action
Bug: trade_order_generators.js set skipAdditionalParsing=true which
caused json_base_generator.js to return early before processing <next>
sibling blocks. This meant set_flag and other blocks connected after
trade_action were silently dropped from the JSON.

Fix: Removed the unnecessary skipAdditionalParsing flag. The code path
it was meant to protect was already in an else branch, so the flag
served no purpose and broke <next> block handling.

Also includes minor fixes to Strategies.js and backtesting.js from
earlier session work.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 02:28:50 -04:00
rob 3976fc8366 Fix backtest data source and chart view detection bugs
- Fix market/symbol key mismatch in PythonGenerator.py (6 locations)
  causing backtest to use wrong trading pair (BTC/USD vs BTC/USDT)
- Fix backtesting.py to always use default_source for backtest data
- Fix exchange/exchange_name key mismatch in app.py and BrighterTrades.py
  causing strategy dialog to show wrong current chart exchange
- Add favicon links to standalone HTML templates
- Add AI strategy dialog template
- Update tests and documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 00:05:31 -04:00
rob 0e481e653e Integrate EDM for candle data and improve UI dialogs
EDM Integration:
- Add edm_client module for REST/WebSocket communication with exchange-data-manager
- Refactor candles.py to fetch data exclusively from EDM
- Add EDM session management for authenticated exchange access
- Update backtesting to use EDM for historical data
- Pass edm_client through to all strategy instances
- Add /api/edm-config, /api/chart-view, /health/edm endpoints
- Remove legacy candle-fetching code from DataCache_v3 and Exchange

UI Improvements:
- Make all popup dialogs draggable and resizable (exchange config, indicator, signal, trade)
- Add default trading source (exchange, symbol, timeframe) to strategy dialog
- Convert symbol fields to dropdowns with smart prioritization (popular pairs first)
- Fix signal dialog property dropdown initialization

Bug Fixes:
- Fix PythonGenerator regex to handle symbols starting with numbers (e.g., 0G/USDT)
- Fix EDM client date format (use Unix timestamps instead of ISO strings)
- Fix strategy default_source database migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 11:42:06 -04:00
rob 33de4affb4 Fix auth UX, exchange config, and trade validation
Auth/Login improvements:
- Add dedicated login page (login_page.html)
- Redirect to /app after successful login/signup instead of welcome
- Add flash message categories to prevent cross-page message pollution
- Filter flash messages by category in templates

Exchange configuration fixes:
- Fix JS discarding API keys for public exchanges (now optional, not cleared)
- Add explicit key/secret validation in backend before saving
- Properly update configured_exchanges when keys provided

Trade dialog enhancements:
- Add exchange selector populated from connected exchanges
- Add testnet checkbox with exchange support detection
- Add production trading confirmation dialog
- Show TESTNET/LIVE badges on trade cards
- Validate exchange is configured before creating live trades

Bug fixes:
- Add try/except around get_user_data in is_logged_in()
- Fix DataFrame index access for user data retrieval

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 20:59:07 -04:00
rob 45395c86c5 Implement fee fetching from exchange via ccxt
Exchange.py:
- Add get_trading_fees(symbol) - fetches fees from market data
  - Tries user-specific fees for authenticated users (fetch_trading_fees)
  - Falls back to market data (public)
  - Returns maker/taker rates with source indicator
- Add get_margin_info(symbol) - returns margin trading availability

ExchangeInterface.py:
- Add get_trading_fees() - routes to appropriate exchange
- Add get_margin_info() - routes to appropriate exchange
- Both methods handle user/exchange lookup with fallback to defaults

trade.py:
- Update new_trade() to fetch actual fees from exchange
- Uses taker fee for market orders, maker fee for limit orders
- Falls back to exchange_fees defaults if fetch fails

Fees now come from actual exchange data (0.1% for Binance spot)
instead of hardcoded defaults.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 16:30:19 -04:00
rob b9730b3a1d Upgrade ccxt from 4.4.8 to 4.5.40
Key improvements:
- 80-190% performance gains in utility functions
- New margin/futures WebSocket support for Binance
- Better liquidation and position tracking
- coincurve library for faster ECDSA signing
- New exchange support (Aster)

All Exchange methods verified working with new version.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 16:19:08 -04:00
rob f603728080 Fix paper trade P/L updates and fee calculations
- Fix default fee from 0.1 (10%) to 0.001 (0.1%) in Trade class
- Add real-time trade updates via WebSocket trade_update events
- Fetch current market price for market orders instead of using cached price
- Add trade persistence to database with proper schema
- Add execution loop to update trades even without active strategies
- Add frontend handler for trade_update events in communication.js
- Add handleTradeUpdate method in trade.js for live P/L updates
- Add debug file logging for trade update debugging
- Update statistics dashboard and trading HUD templates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 16:05:53 -04:00
rob 78dfd71303 Add statistics dashboard and strategy UI enhancements
Statistics HUD:
- Full statistics dashboard with running strategies list
- Strategy performance metrics (P&L, balance, trades, win rate)
- Mini equity curve chart placeholder
- Real-time stats updates via SocketIO

Strategy Improvements:
- Orphaned strategy cleanup function
- User existence validation
- Enhanced strategy card display
- Improved new strategy popup

UI/UX Enhancements:
- Updated control panel layout
- Indicator blocks improvements
- Additional CSS styling
- Welcome template
- Communication.js strategy event handling

Backend:
- Strategy instance tracking updates
- Paper strategy instance improvements
- User session handling updates
- App.py route additions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 04:39:41 -04:00
rob c895a3615d Refactor Signals system and implement Alerts
Signals System Refactor:
- Migrated from config.yml to DataCache/SQLite storage
- Added user ownership (creator field) and public/private visibility
- Implemented full CRUD operations with tbl_key identifiers
- Auto-creates signals table in database on init
- Card-based UI with 100x100 icons and hover detail panels
- State visualization: green pulsing border (TRUE), red (FALSE)
- Edit support via popup form with Create/Edit buttons

Alerts System Implementation:
- Connected frontend Alerts.js to SocketIO communications
- Alerts triggered by signal state changes (s_updates)
- Support for strategy events, trades, errors, notifications
- Color-coded alerts with icons and timestamps
- Clear button and max 50 alerts limit (ephemeral, not persisted)

Backend changes:
- BrighterTrades.py: Updated signal handlers with user_id
- Added received_edit_signal() and permission checks
- Signal filtering by user ownership

Frontend changes:
- signals.js: Three-manager pattern (SigUIManager, SigDataManager, Signals)
- Alerts.js: Event handlers for updates and strategy_events
- general.js: Initialize signals and alerts with comms

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 04:31:39 -04:00
rob dd8467468c Fix Binance API error -2015 when fetching positions on spot exchanges
- Check if exchange supports fetchPositions before calling it
- Handle PermissionDenied (error -2015) gracefully with debug log instead of error
- Handle NotSupported exception for exchanges that don't support positions
- Use ccxt standard field names (contracts, entryPrice) with fallbacks
- Skip positions with zero size
- Add documentation noting this is a futures/derivatives feature

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 18:03:55 -04:00
rob 3644c219d4 Fix int64 JSON serialization error in SocketIO responses
- Add utils.py with sanitize_for_json() function to convert numpy types
  (int64, float64, bool_, ndarray) to native Python types
- Handle inf/nan values by converting to None
- Update backtesting.py to use shared sanitize_for_json utility
- Update app.py strategy_events emit to sanitize data before sending
- Remove empty root BrighterTrading.db and add *.db to gitignore

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 17:58:24 -04:00
rob 4e14463465 Add user feedback for exchange configuration
- Show loading state while connecting to exchange
- Display success/error messages in the form
- Prevent double-submission with disabled button
- Add 30-second timeout with error message
- Register handler for Exchange_connection_result message
- Reload page after successful connection (with delay to show message)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 17:34:27 -04:00
rob da9d1fc5ee Fix test_BrighterTrades.py socketio mock issue
- Added proper mocking for all BrighterTrades dependencies (DataCache,
  ExchangeInterface, Users, Strategies, Candles, Indicators, Trades)
- Pass mock socketio instance to BrighterTrades constructor
- Fixed patch paths to use correct module path (src.BrighterTrades.*)
- All 5 tests now pass

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 17:31:20 -04:00
rob 639043b261 Live trading infrastructure functional on testnet
Key fixes:
- ExchangeInterface: Remove stale entries before creating new exchange connections
- LiveBroker: Optimize get_total_equity() to price only top 10 assets (was hanging on 462 testnet assets)
- LiveBroker: Add fiat currency skip list to avoid failed price lookups
- PythonGenerator: Fix market symbol regex to handle 2-5 char symbols (BTC/USDT)

New features:
- LiveStrategyInstance: Full live trading strategy execution
- Circuit breaker and position limits for live trading safety
- Restart-safe order reconciliation via client order IDs

Verified working:
- Live strategy starts on Binance testnet
- Orders placed and filled successfully
- Execution loop runs with active strategies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 15:23:34 -04:00
rob 6821f821e1 Implement execution loop, paper persistence, and CI pipeline
Strategy Execution Loop:
- Add tick() method to StrategyInstance for price-driven execution
- Add update() method to Strategies for iterating active instances
- Enable execution loop in received_cdata() to process candle updates
- Add PaperStrategyInstance.tick() with broker price updates

Paper Trading Persistence:
- Add Position.to_dict()/from_dict() for serialization
- Add PaperBroker state persistence (save_state/load_state)
- Add _ensure_persistence_cache() with DB schema migration
- Auto-load/save broker state in PaperStrategyInstance

Runtime Fixes (from Codex review):
- Fix get_user_info() signature mismatch in start_strategy
- Fix live/paper mode handling for stop operations
- Normalize fill event payload (filled_qty/filled_price keys)
- Remove double broker update path (delegate to tick)
- Prevent runtime events from polluting strategy list in UI

CI Pipeline:
- Add GitHub Actions workflow (.github/workflows/test.yml)
- Python 3.12 with TA-Lib dependency
- Syntax checks and 5 critical test suites

Tests: 85 passed
- test_strategy_execution.py (16 tests)
- test_execution_loop.py (17 tests)
- test_paper_persistence.py (19 tests)
- test_backtest_determinism.py (13 tests)
- test_brokers.py (18 tests)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 19:48:50 -04:00
rob b555f6e004 Fix high-severity issues identified by Codex
1. Paper equity calculation (High):
   - get_balance() now correctly includes position value (size * current_price)
   - Previously only added unrealized_pnl, missing position principal

2. Buy affordability checks (High):
   - place_order() now uses execution price including slippage for market orders
   - Uses limit price for limit orders instead of current price
   - Prevents orders that would drive cash negative

3. ExchangeInterface price stubs (High):
   - get_trade_executed_price() now handles order_price=0.0 (market orders)
   - Falls back to current price or entry price when order_price is zero

4. Determinism hash strengthening (Medium):
   - Hash now includes full trade sequence with individual trade details
   - Added test for different trade sequences with same totals

5. Live mode consistency (Medium):
   - Factory now falls back to paper trading with warning instead of raising
   - Consistent behavior between factory.py and Strategies.py

Added test: test_paper_broker_equity_calculation to verify fix #1

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 17:23:58 -04:00
rob 5dbda13924 Phase 5-6: Live trading stubs and observability
Phase 5 - Live Trading (stub implementation):
- Create LiveBroker stub with NotImplementedError for all methods
- Document required exchange API integration points
- Add testnet flag for safety

Phase 6 - Observability:
- Add structured logging with StructuredFormatter and ColoredFormatter
- Create TradingLogger for trading-specific log entries
- Implement health check system with HealthCheck class
- Add default health checks for database, exchange, memory
- Create health_endpoint() for monitoring integration

The LiveBroker is a stub that needs exchange API integration
for production use. All other trading modes (backtest, paper)
are fully functional.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 17:09:54 -04:00
rob 51ec74175d Phase 4: Paper trading implementation
- Create PaperStrategyInstance extending StrategyInstance
- Integrate PaperBroker for simulated order execution
- Add trade_order() method translating Blockly calls to broker
- Add mode selection in Strategies.create_strategy_instance()
- Update execute_strategy() to support paper/backtest/live modes
- Include comprehensive tests for paper trading functionality

Paper trading now works with:
- Market and limit orders
- Position tracking with P&L
- Balance management
- Trade history
- Reset functionality

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 17:07:34 -04:00
rob 1bb224b15d Phase 3: Backtest determinism
- Fix deprecated datetime.utcfromtimestamp() in shared_utilities.py
- Create BacktestResult schema with locked structure
- Add TradeResult and BacktestMetrics dataclasses
- Implement determinism hash for verifying reproducible results
- Add comprehensive tests for result schema and determinism

The BacktestResult schema ensures consistent output format and
provides methods to verify that same inputs produce same results.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 17:02:54 -04:00
rob f1182d4e0c Phase 2: Broker abstraction foundation
- Create brokers package with unified trading interface
- BaseBroker: Abstract base class defining broker contract
- BacktestBroker: Wraps Backtrader for backtesting mode
- PaperBroker: Simulated order execution for paper trading
- Factory function to create broker based on trading mode
- Comprehensive test suite for broker functionality

The broker abstraction enables strategies to work identically
across backtest, paper, and live trading modes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 17:00:43 -04:00
rob e245f80e18 Phase 1: Repair runtime breakpoints
- Fix set_new_candle() to update cached candle instead of raising
- Fix trades.update() type mismatch: pass {symbol: price} dict
- Fix StrategyInstance.trades collision: rename list to trade_history
- Add missing ExchangeInterface methods for trade status/qty/price
- Add get_user_balance stub for paper trading

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 16:47:13 -04:00
rob 34637df478 Phase 0: Stabilize baseline
- Security: Move API keys to environment variables in config.py
- Portability: Use cross-platform path resolution for DB_FILE
- Add config.example.py template for developers
- Fix Windows path in ExchangeInterface.get_public_exchanges()
- Add cached_last_candle attribute to Candles class
- Add pytest configuration (pytest.ini, conftest.py)
- Fix test imports for DataCache_v3
- Include identity compatibility layer (user_name/user_id resolution)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 16:44:11 -04:00
rob f1a184b131 Fix chart loading and stabilize local runtime 2026-02-28 15:15:56 -04:00
Rob ee51ab1d8c I am just about to move most of the strategy statistics tracking to the StrategyInstance 2024-11-22 10:28:39 -04:00
Rob dab122d15f A lot of stuff is fixed. The strategies need to reflect the stats in the ui. 2024-11-20 01:38:46 -04:00
Rob e1516a80cf A lot of stuff is fixed. only issues at the moment are that errors during testing are not displayed in the ui and the strategies need to reflect the stats. 2024-11-20 00:18:01 -04:00
Rob 2c644147a4 I repaired the sinup and signin parts. I changed a bunch of javascript in strategies.js. there is an error that pops up when webtools is open. 2024-11-18 10:25:08 -04:00
Rob fe1e93c24b The test seem to run without errors. It now yields to other processes. 2024-11-15 19:51:10 -04:00
Rob 4072a9d5f5 The test seem to run without errors but are still process blocking. 2024-11-15 18:08:32 -04:00
Rob 4f11778b09 The test are running and returning feedback. I am just about to implement a better solution for multithreading 2024-11-13 22:15:35 -04:00
Rob 0ae835d096 The test are displaying in the user interface. 2024-11-13 01:13:29 -04:00
Rob 1af55f10a0 The test are running without errors 2024-11-12 09:43:18 -04:00
Rob 33298b7178 The test are running but not without errors. 2024-11-11 17:22:06 -04:00
Rob 4fcc6f661d Python generators is fully implemented but not tested yet. 2024-11-07 10:43:51 -04:00
Rob 9d830fe8fa Strtegies submit an are editable. 2024-11-05 12:02:50 -04:00
Rob 50bcfa05cd The generator definitions seem to work now. 2024-11-04 18:20:09 -04:00
Rob bb7fdbf8f9 The generator definitions are still being tested but progress has been made. 2024-10-27 16:51:10 -03:00
Rob 5110c8b434 I adjusted the block definitions and expanded them. The generator definitions have been updated but are not tested. 2024-10-21 13:37:05 -03:00
Rob c7b7a47129 I played around with the blocks and i think i have it correct now. 2024-10-16 00:21:43 -03:00
Rob 232d479827 nearly all core functionality is flushed out. 2024-10-13 01:23:26 -03:00
Rob 4eda0b6f81 The strategies class is broken into smaller classes. I believe the ui is mostly functional at this point. It is time to get the tests working. 2024-10-10 17:05:07 -03:00
Rob 89e0f8b849 I decoupled the comms class from a few other classes. I updated some of the ui in regard to this indicator readouts. An just about to attempt to break up the strategies class. 2024-10-09 09:09:52 -03:00
Rob 86843e8cb4 Strategies is now fixed and I am ready to implement backtesting. 2024-09-25 14:27:56 -03:00
Rob adedaaa540 Getting closer to a beta. woot woot. there is a problem with created strategies only displaying two of the ones in the database 2024-09-24 23:43:13 -03:00
Rob 4e3e8e5abf The strategy popup is now draggable and resizeable. 2024-09-23 16:38:22 -03:00
Rob 034b8ab925 The strategy section is generating code. Implemented code is mostly functional. 2024-09-23 09:55:27 -03:00
Rob 38de03c022 I think it is all working except I don't think the indicators are caching on the database. 2024-09-20 17:40:47 -03:00