Commit Graph

26 Commits

Author SHA1 Message Date
rob b4bf6c5c8e Add external data sources and indicators for backtesting
External Sources:
- New ExternalSources.py module for real-time API data fetching
- Configurable refresh intervals, JSONPath extraction, and auth headers
- External sources can be used in signals for live strategy conditions

External Indicators:
- New ExternalIndicators.py module for historical API data
- Supports backtesting by fetching and caching historical data points
- Timestamps are matched to candle times during backtest execution
- Fear & Greed index and similar APIs now work in backtests

Signal System Enhancements:
- Added signal Blockly blocks for use in strategy builder
- New signal_blocks.js for dynamic signal block generation
- Signals category added to strategy toolbox
- PythonGenerator updated to handle signal_* block types
- process_signal() method added to StrategyInstance classes

notify_user Block Enhancement:
- Now supports dynamic values (variables, indicators, expressions)
- Can display indicator values and calculations in notifications

Bug Fixes:
- Fixed Socket.IO parse error causing strategies not to load on refresh
- Enhanced sanitize_for_json to handle pandas, datetime, bytes types
- Fixed external indicator name matching (underscore/space conversion)
- Fixed PRAGMA query missing fetch_all=True in ExternalIndicators

Dependencies:
- Added jsonpath-ng for JSONPath extraction from API responses

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-09 21:25:36 -03:00
rob 7e77f55837 Add Bitcoin wallet system with configurable strategy fees
Features:
- Two-address custody model (Fee Address + Sweep Address)
- Credits ledger for instant, reliable strategy fee transactions
- Configurable fee percentage (1-100% of exchange commission)
- Background jobs for auto-sweep, deposit detection, withdrawal processing
- Account settings dialog accessible via username click
- $50 balance cap with auto-sweep to user's sweep address

Security improvements:
- Atomic withdrawal reservation prevents partial state
- Fee accumulation cleanup on strategy startup failure
- Deposit monitoring includes disabled wallets for recovery
- Null tx hash checks prevent silent failures
- Key export disabled by default

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-09 12:45:25 -03:00
rob cd9a69f1d4 Add exchange validation, fix indicators bug, and improve balance display
- Add exchange_validation.py module for validating exchange requirements
  before running strategies (backtest, paper, live modes)
- Fix AttributeError in Signals.py: 'Indicators' object has no attribute
  'indicators' - created IndicatorWrapper class for proper data access
- Fix testnet balance issue: explicitly pass testnet=False to all
  connect_exchange calls to prevent pickle corruption from old testnet
  Exchange objects
- Add balance exchange selector: display one exchange at a time with
  dropdown to switch between connected exchanges (defaults to chart view)
- Add unique tbl_key generation for exchange_data to prevent duplicate
  entries (format: user:exchange)
- Fix DataCache balance serialization for list types
- Update frontend error handling for exchange validation errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-08 19:41:41 -03:00
rob ee16023b6b Add public strategy subscription system
Implement a subscription system for public strategies:

Security & Auth:
- Bind WebSocket user identity at connect time (prevents spoofing)
- Add server-side ownership verification for all destructive operations
- Invalidate socket connections on logout
- Add XSS protection with HTML/JS escaping in frontend

Database:
- Add strategy_subscriptions table with proper indexes
- Fix get_all_rows_from_datacache to fall back to DB when cache empty

Backend:
- Add subscribe/unsubscribe endpoints with authorization checks
- Add get_user_strategies (owned + subscribed) and get_public_strategies_catalog
- Propagate indicator_owner_id through strategy instances for subscribed strategies
- Redact strategy internals (code, workspace) for non-owners

Frontend:
- Add "Add Public" button to browse and subscribe to public strategies
- Show subscribed strategies with creator badge and unsubscribe button
- Prevent editing of subscribed strategies (show info modal instead)
- Add public strategy browser modal

Tests:
- Update authorization tests for new subscription-required model

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-08 16:59:52 -03:00
rob 91f51cd71f Fix set_flag blocks being dropped after trade_action
Bug: trade_order_generators.js set skipAdditionalParsing=true which
caused json_base_generator.js to return early before processing <next>
sibling blocks. This meant set_flag and other blocks connected after
trade_action were silently dropped from the JSON.

Fix: Removed the unnecessary skipAdditionalParsing flag. The code path
it was meant to protect was already in an else branch, so the flag
served no purpose and broke <next> block handling.

Also includes minor fixes to Strategies.js and backtesting.js from
earlier session work.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 02:28:50 -04:00
rob 3976fc8366 Fix backtest data source and chart view detection bugs
- Fix market/symbol key mismatch in PythonGenerator.py (6 locations)
  causing backtest to use wrong trading pair (BTC/USD vs BTC/USDT)
- Fix backtesting.py to always use default_source for backtest data
- Fix exchange/exchange_name key mismatch in app.py and BrighterTrades.py
  causing strategy dialog to show wrong current chart exchange
- Add favicon links to standalone HTML templates
- Add AI strategy dialog template
- Update tests and documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 00:05:31 -04:00
rob 0e481e653e Integrate EDM for candle data and improve UI dialogs
EDM Integration:
- Add edm_client module for REST/WebSocket communication with exchange-data-manager
- Refactor candles.py to fetch data exclusively from EDM
- Add EDM session management for authenticated exchange access
- Update backtesting to use EDM for historical data
- Pass edm_client through to all strategy instances
- Add /api/edm-config, /api/chart-view, /health/edm endpoints
- Remove legacy candle-fetching code from DataCache_v3 and Exchange

UI Improvements:
- Make all popup dialogs draggable and resizable (exchange config, indicator, signal, trade)
- Add default trading source (exchange, symbol, timeframe) to strategy dialog
- Convert symbol fields to dropdowns with smart prioritization (popular pairs first)
- Fix signal dialog property dropdown initialization

Bug Fixes:
- Fix PythonGenerator regex to handle symbols starting with numbers (e.g., 0G/USDT)
- Fix EDM client date format (use Unix timestamps instead of ISO strings)
- Fix strategy default_source database migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 11:42:06 -04:00
rob 78dfd71303 Add statistics dashboard and strategy UI enhancements
Statistics HUD:
- Full statistics dashboard with running strategies list
- Strategy performance metrics (P&L, balance, trades, win rate)
- Mini equity curve chart placeholder
- Real-time stats updates via SocketIO

Strategy Improvements:
- Orphaned strategy cleanup function
- User existence validation
- Enhanced strategy card display
- Improved new strategy popup

UI/UX Enhancements:
- Updated control panel layout
- Indicator blocks improvements
- Additional CSS styling
- Welcome template
- Communication.js strategy event handling

Backend:
- Strategy instance tracking updates
- Paper strategy instance improvements
- User session handling updates
- App.py route additions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 04:39:41 -04:00
rob 639043b261 Live trading infrastructure functional on testnet
Key fixes:
- ExchangeInterface: Remove stale entries before creating new exchange connections
- LiveBroker: Optimize get_total_equity() to price only top 10 assets (was hanging on 462 testnet assets)
- LiveBroker: Add fiat currency skip list to avoid failed price lookups
- PythonGenerator: Fix market symbol regex to handle 2-5 char symbols (BTC/USDT)

New features:
- LiveStrategyInstance: Full live trading strategy execution
- Circuit breaker and position limits for live trading safety
- Restart-safe order reconciliation via client order IDs

Verified working:
- Live strategy starts on Binance testnet
- Orders placed and filled successfully
- Execution loop runs with active strategies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 15:23:34 -04:00
rob 6821f821e1 Implement execution loop, paper persistence, and CI pipeline
Strategy Execution Loop:
- Add tick() method to StrategyInstance for price-driven execution
- Add update() method to Strategies for iterating active instances
- Enable execution loop in received_cdata() to process candle updates
- Add PaperStrategyInstance.tick() with broker price updates

Paper Trading Persistence:
- Add Position.to_dict()/from_dict() for serialization
- Add PaperBroker state persistence (save_state/load_state)
- Add _ensure_persistence_cache() with DB schema migration
- Auto-load/save broker state in PaperStrategyInstance

Runtime Fixes (from Codex review):
- Fix get_user_info() signature mismatch in start_strategy
- Fix live/paper mode handling for stop operations
- Normalize fill event payload (filled_qty/filled_price keys)
- Remove double broker update path (delegate to tick)
- Prevent runtime events from polluting strategy list in UI

CI Pipeline:
- Add GitHub Actions workflow (.github/workflows/test.yml)
- Python 3.12 with TA-Lib dependency
- Syntax checks and 5 critical test suites

Tests: 85 passed
- test_strategy_execution.py (16 tests)
- test_execution_loop.py (17 tests)
- test_paper_persistence.py (19 tests)
- test_backtest_determinism.py (13 tests)
- test_brokers.py (18 tests)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 19:48:50 -04:00
Rob e1516a80cf A lot of stuff is fixed. only issues at the moment are that errors during testing are not displayed in the ui and the strategies need to reflect the stats. 2024-11-20 00:18:01 -04:00
Rob 2c644147a4 I repaired the sinup and signin parts. I changed a bunch of javascript in strategies.js. there is an error that pops up when webtools is open. 2024-11-18 10:25:08 -04:00
Rob 9d830fe8fa Strtegies submit an are editable. 2024-11-05 12:02:50 -04:00
Rob 50bcfa05cd The generator definitions seem to work now. 2024-11-04 18:20:09 -04:00
Rob bb7fdbf8f9 The generator definitions are still being tested but progress has been made. 2024-10-27 16:51:10 -03:00
Rob 5110c8b434 I adjusted the block definitions and expanded them. The generator definitions have been updated but are not tested. 2024-10-21 13:37:05 -03:00
Rob c7b7a47129 I played around with the blocks and i think i have it correct now. 2024-10-16 00:21:43 -03:00
Rob 232d479827 nearly all core functionality is flushed out. 2024-10-13 01:23:26 -03:00
Rob 4eda0b6f81 The strategies class is broken into smaller classes. I believe the ui is mostly functional at this point. It is time to get the tests working. 2024-10-10 17:05:07 -03:00
Rob 89e0f8b849 I decoupled the comms class from a few other classes. I updated some of the ui in regard to this indicator readouts. An just about to attempt to break up the strategies class. 2024-10-09 09:09:52 -03:00
Rob 86843e8cb4 Strategies is now fixed and I am ready to implement backtesting. 2024-09-25 14:27:56 -03:00
Rob adedaaa540 Getting closer to a beta. woot woot. there is a problem with created strategies only displaying two of the ones in the database 2024-09-24 23:43:13 -03:00
Rob 4e3e8e5abf The strategy popup is now draggable and resizeable. 2024-09-23 16:38:22 -03:00
Rob 034b8ab925 The strategy section is generating code. Implemented code is mostly functional. 2024-09-23 09:55:27 -03:00
Rob fc407708ba Fixed an issue inserting candles into the database. I realized I want to get rid of the id column in the market(candles) tables. Began integrating Blockly into the client. 2024-08-17 18:10:34 -03:00
Rob 0bb9780ff6 Fixed issue with fetching historical data and added missing imports. 2024-07-26 03:52:24 -03:00