Commit Graph

40 Commits

Author SHA1 Message Date
rob 5717ac6a81 Implement chart formations MVP with Line and Channel support
Add formations feature for drawing trendlines and channels on charts:

Backend (Formations.py):
- CRUD operations with DataCache integration
- Database table with scope-based indexing
- Line value calculation with infinite extension
- Property value lookup for strategy integration

Frontend (formations.js, formation_overlay.js):
- Three-class pattern: UIManager, DataManager, Controller
- SVG overlay for drawing with RAF sync loop
- Click-to-draw interface with temp line preview
- Anchor dragging for line adjustment
- Coordinate conversion using v5 API

UI Integration:
- HUD panel with Line/Channel buttons
- Formation cards with hover details
- Drawing controls (name input, save/cancel)

Socket handlers for real-time sync:
- request_formations, new_formation
- edit_formation, delete_formation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-10 16:22:08 -03:00
rob b4bf6c5c8e Add external data sources and indicators for backtesting
External Sources:
- New ExternalSources.py module for real-time API data fetching
- Configurable refresh intervals, JSONPath extraction, and auth headers
- External sources can be used in signals for live strategy conditions

External Indicators:
- New ExternalIndicators.py module for historical API data
- Supports backtesting by fetching and caching historical data points
- Timestamps are matched to candle times during backtest execution
- Fear & Greed index and similar APIs now work in backtests

Signal System Enhancements:
- Added signal Blockly blocks for use in strategy builder
- New signal_blocks.js for dynamic signal block generation
- Signals category added to strategy toolbox
- PythonGenerator updated to handle signal_* block types
- process_signal() method added to StrategyInstance classes

notify_user Block Enhancement:
- Now supports dynamic values (variables, indicators, expressions)
- Can display indicator values and calculations in notifications

Bug Fixes:
- Fixed Socket.IO parse error causing strategies not to load on refresh
- Enhanced sanitize_for_json to handle pandas, datetime, bytes types
- Fixed external indicator name matching (underscore/space conversion)
- Fixed PRAGMA query missing fetch_all=True in ExternalIndicators

Dependencies:
- Added jsonpath-ng for JSONPath extraction from API responses

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-09 21:25:36 -03:00
rob 7e77f55837 Add Bitcoin wallet system with configurable strategy fees
Features:
- Two-address custody model (Fee Address + Sweep Address)
- Credits ledger for instant, reliable strategy fee transactions
- Configurable fee percentage (1-100% of exchange commission)
- Background jobs for auto-sweep, deposit detection, withdrawal processing
- Account settings dialog accessible via username click
- $50 balance cap with auto-sweep to user's sweep address

Security improvements:
- Atomic withdrawal reservation prevents partial state
- Fee accumulation cleanup on strategy startup failure
- Deposit monitoring includes disabled wallets for recovery
- Null tx hash checks prevent silent failures
- Key export disabled by default

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-09 12:45:25 -03:00
rob cd9a69f1d4 Add exchange validation, fix indicators bug, and improve balance display
- Add exchange_validation.py module for validating exchange requirements
  before running strategies (backtest, paper, live modes)
- Fix AttributeError in Signals.py: 'Indicators' object has no attribute
  'indicators' - created IndicatorWrapper class for proper data access
- Fix testnet balance issue: explicitly pass testnet=False to all
  connect_exchange calls to prevent pickle corruption from old testnet
  Exchange objects
- Add balance exchange selector: display one exchange at a time with
  dropdown to switch between connected exchanges (defaults to chart view)
- Add unique tbl_key generation for exchange_data to prevent duplicate
  entries (format: user:exchange)
- Fix DataCache balance serialization for list types
- Update frontend error handling for exchange validation errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-08 19:41:41 -03:00
rob ee16023b6b Add public strategy subscription system
Implement a subscription system for public strategies:

Security & Auth:
- Bind WebSocket user identity at connect time (prevents spoofing)
- Add server-side ownership verification for all destructive operations
- Invalidate socket connections on logout
- Add XSS protection with HTML/JS escaping in frontend

Database:
- Add strategy_subscriptions table with proper indexes
- Fix get_all_rows_from_datacache to fall back to DB when cache empty

Backend:
- Add subscribe/unsubscribe endpoints with authorization checks
- Add get_user_strategies (owned + subscribed) and get_public_strategies_catalog
- Propagate indicator_owner_id through strategy instances for subscribed strategies
- Redact strategy internals (code, workspace) for non-owners

Frontend:
- Add "Add Public" button to browse and subscribe to public strategies
- Show subscribed strategies with creator badge and unsubscribe button
- Prevent editing of subscribed strategies (show info modal instead)
- Add public strategy browser modal

Tests:
- Update authorization tests for new subscription-required model

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-08 16:59:52 -03:00
rob 7b796c51c8 Fix indicator toggle functionality and add loading feedback
- Fix JSON parsing bug: JavaScript FormData sends indicator list as
  JSON string inside a list, now properly parsed on backend
- Move form event listener setup to _setupIndicatorForm() called from
  addToCharts() so it's always attached regardless of indicator data
- Add immediate visual feedback: popup hides instantly on submit and
  button shows "Updating..." text

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-08 03:33:45 -03:00
rob 3e6463e4b3 Add Bollinger %B indicator and fix credential handling
- Add BollingerPercentB indicator class in backend (BOL%B)
  - Calculates %B = (Close - Lower) / (Upper - Lower)
  - Shows where price is relative to Bollinger Bands
  - Values: 1.0 = upper band, 0.5 = middle, 0.0 = lower
- Add BollingerPercentB JavaScript class for frontend display
  - Creates its own chart (chart4) like RSI/MACD
  - Includes chart binding for synchronized scrolling
- Add chart4 HTML container and data plumbing
- Fix credential update when reconnecting with changed API keys
- Add TESTNET_MODE config setting for live trading mode control

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 06:31:31 -04:00
rob 21d449d877 Fix KuCoin balance fetching and add balance refresh functionality
- Add KuCoin-specific balance fetching with type='trade' parameter
- Reinitialize ccxt client in refresh_balances() to fix pickle corruption
- Force reconnection when exchange is restored from database cache
- Add balance refresh button and socket handler in frontend
- Fix template null check for balances display
- Clean up DataCache and candles imports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 06:10:31 -04:00
rob 3976fc8366 Fix backtest data source and chart view detection bugs
- Fix market/symbol key mismatch in PythonGenerator.py (6 locations)
  causing backtest to use wrong trading pair (BTC/USD vs BTC/USDT)
- Fix backtesting.py to always use default_source for backtest data
- Fix exchange/exchange_name key mismatch in app.py and BrighterTrades.py
  causing strategy dialog to show wrong current chart exchange
- Add favicon links to standalone HTML templates
- Add AI strategy dialog template
- Update tests and documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-06 00:05:31 -04:00
rob 0e481e653e Integrate EDM for candle data and improve UI dialogs
EDM Integration:
- Add edm_client module for REST/WebSocket communication with exchange-data-manager
- Refactor candles.py to fetch data exclusively from EDM
- Add EDM session management for authenticated exchange access
- Update backtesting to use EDM for historical data
- Pass edm_client through to all strategy instances
- Add /api/edm-config, /api/chart-view, /health/edm endpoints
- Remove legacy candle-fetching code from DataCache_v3 and Exchange

UI Improvements:
- Make all popup dialogs draggable and resizable (exchange config, indicator, signal, trade)
- Add default trading source (exchange, symbol, timeframe) to strategy dialog
- Convert symbol fields to dropdowns with smart prioritization (popular pairs first)
- Fix signal dialog property dropdown initialization

Bug Fixes:
- Fix PythonGenerator regex to handle symbols starting with numbers (e.g., 0G/USDT)
- Fix EDM client date format (use Unix timestamps instead of ISO strings)
- Fix strategy default_source database migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 11:42:06 -04:00
rob 33de4affb4 Fix auth UX, exchange config, and trade validation
Auth/Login improvements:
- Add dedicated login page (login_page.html)
- Redirect to /app after successful login/signup instead of welcome
- Add flash message categories to prevent cross-page message pollution
- Filter flash messages by category in templates

Exchange configuration fixes:
- Fix JS discarding API keys for public exchanges (now optional, not cleared)
- Add explicit key/secret validation in backend before saving
- Properly update configured_exchanges when keys provided

Trade dialog enhancements:
- Add exchange selector populated from connected exchanges
- Add testnet checkbox with exchange support detection
- Add production trading confirmation dialog
- Show TESTNET/LIVE badges on trade cards
- Validate exchange is configured before creating live trades

Bug fixes:
- Add try/except around get_user_data in is_logged_in()
- Fix DataFrame index access for user data retrieval

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 20:59:07 -04:00
rob f603728080 Fix paper trade P/L updates and fee calculations
- Fix default fee from 0.1 (10%) to 0.001 (0.1%) in Trade class
- Add real-time trade updates via WebSocket trade_update events
- Fetch current market price for market orders instead of using cached price
- Add trade persistence to database with proper schema
- Add execution loop to update trades even without active strategies
- Add frontend handler for trade_update events in communication.js
- Add handleTradeUpdate method in trade.js for live P/L updates
- Add debug file logging for trade update debugging
- Update statistics dashboard and trading HUD templates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 16:05:53 -04:00
rob c895a3615d Refactor Signals system and implement Alerts
Signals System Refactor:
- Migrated from config.yml to DataCache/SQLite storage
- Added user ownership (creator field) and public/private visibility
- Implemented full CRUD operations with tbl_key identifiers
- Auto-creates signals table in database on init
- Card-based UI with 100x100 icons and hover detail panels
- State visualization: green pulsing border (TRUE), red (FALSE)
- Edit support via popup form with Create/Edit buttons

Alerts System Implementation:
- Connected frontend Alerts.js to SocketIO communications
- Alerts triggered by signal state changes (s_updates)
- Support for strategy events, trades, errors, notifications
- Color-coded alerts with icons and timestamps
- Clear button and max 50 alerts limit (ephemeral, not persisted)

Backend changes:
- BrighterTrades.py: Updated signal handlers with user_id
- Added received_edit_signal() and permission checks
- Signal filtering by user ownership

Frontend changes:
- signals.js: Three-manager pattern (SigUIManager, SigDataManager, Signals)
- Alerts.js: Event handlers for updates and strategy_events
- general.js: Initialize signals and alerts with comms

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 04:31:39 -04:00
rob 639043b261 Live trading infrastructure functional on testnet
Key fixes:
- ExchangeInterface: Remove stale entries before creating new exchange connections
- LiveBroker: Optimize get_total_equity() to price only top 10 assets (was hanging on 462 testnet assets)
- LiveBroker: Add fiat currency skip list to avoid failed price lookups
- PythonGenerator: Fix market symbol regex to handle 2-5 char symbols (BTC/USDT)

New features:
- LiveStrategyInstance: Full live trading strategy execution
- Circuit breaker and position limits for live trading safety
- Restart-safe order reconciliation via client order IDs

Verified working:
- Live strategy starts on Binance testnet
- Orders placed and filled successfully
- Execution loop runs with active strategies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 15:23:34 -04:00
rob 6821f821e1 Implement execution loop, paper persistence, and CI pipeline
Strategy Execution Loop:
- Add tick() method to StrategyInstance for price-driven execution
- Add update() method to Strategies for iterating active instances
- Enable execution loop in received_cdata() to process candle updates
- Add PaperStrategyInstance.tick() with broker price updates

Paper Trading Persistence:
- Add Position.to_dict()/from_dict() for serialization
- Add PaperBroker state persistence (save_state/load_state)
- Add _ensure_persistence_cache() with DB schema migration
- Auto-load/save broker state in PaperStrategyInstance

Runtime Fixes (from Codex review):
- Fix get_user_info() signature mismatch in start_strategy
- Fix live/paper mode handling for stop operations
- Normalize fill event payload (filled_qty/filled_price keys)
- Remove double broker update path (delegate to tick)
- Prevent runtime events from polluting strategy list in UI

CI Pipeline:
- Add GitHub Actions workflow (.github/workflows/test.yml)
- Python 3.12 with TA-Lib dependency
- Syntax checks and 5 critical test suites

Tests: 85 passed
- test_strategy_execution.py (16 tests)
- test_execution_loop.py (17 tests)
- test_paper_persistence.py (19 tests)
- test_backtest_determinism.py (13 tests)
- test_brokers.py (18 tests)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 19:48:50 -04:00
rob e245f80e18 Phase 1: Repair runtime breakpoints
- Fix set_new_candle() to update cached candle instead of raising
- Fix trades.update() type mismatch: pass {symbol: price} dict
- Fix StrategyInstance.trades collision: rename list to trade_history
- Add missing ExchangeInterface methods for trade status/qty/price
- Add get_user_balance stub for paper trading

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 16:47:13 -04:00
rob 34637df478 Phase 0: Stabilize baseline
- Security: Move API keys to environment variables in config.py
- Portability: Use cross-platform path resolution for DB_FILE
- Add config.example.py template for developers
- Fix Windows path in ExchangeInterface.get_public_exchanges()
- Add cached_last_candle attribute to Candles class
- Add pytest configuration (pytest.ini, conftest.py)
- Fix test imports for DataCache_v3
- Include identity compatibility layer (user_name/user_id resolution)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 16:44:11 -04:00
Rob dab122d15f A lot of stuff is fixed. The strategies need to reflect the stats in the ui. 2024-11-20 01:38:46 -04:00
Rob e1516a80cf A lot of stuff is fixed. only issues at the moment are that errors during testing are not displayed in the ui and the strategies need to reflect the stats. 2024-11-20 00:18:01 -04:00
Rob 2c644147a4 I repaired the sinup and signin parts. I changed a bunch of javascript in strategies.js. there is an error that pops up when webtools is open. 2024-11-18 10:25:08 -04:00
Rob 9d830fe8fa Strtegies submit an are editable. 2024-11-05 12:02:50 -04:00
Rob 5110c8b434 I adjusted the block definitions and expanded them. The generator definitions have been updated but are not tested. 2024-10-21 13:37:05 -03:00
Rob c7b7a47129 I played around with the blocks and i think i have it correct now. 2024-10-16 00:21:43 -03:00
Rob 232d479827 nearly all core functionality is flushed out. 2024-10-13 01:23:26 -03:00
Rob 4eda0b6f81 The strategies class is broken into smaller classes. I believe the ui is mostly functional at this point. It is time to get the tests working. 2024-10-10 17:05:07 -03:00
Rob 89e0f8b849 I decoupled the comms class from a few other classes. I updated some of the ui in regard to this indicator readouts. An just about to attempt to break up the strategies class. 2024-10-09 09:09:52 -03:00
Rob 86843e8cb4 Strategies is now fixed and I am ready to implement backtesting. 2024-09-25 14:27:56 -03:00
Rob adedaaa540 Getting closer to a beta. woot woot. there is a problem with created strategies only displaying two of the ones in the database 2024-09-24 23:43:13 -03:00
Rob 4e3e8e5abf The strategy popup is now draggable and resizeable. 2024-09-23 16:38:22 -03:00
Rob f1d0f2a4b1 Indicators are fixed after last update. 2024-09-18 13:47:07 -03:00
Rob 1ff21b56dd Made a lot of changes to DataCache indicator data is not being saved to the database. 2024-09-15 14:05:08 -03:00
Rob 73ed1a092a got all the indicators working again 2024-09-08 04:04:58 -03:00
Rob b666ec22af Extended DataCache with indicator caching functionality. All DataCache tests pass. 2024-09-07 19:51:01 -03:00
Rob a16cc542d2 Refactored DataCache, again. Implemented more advance cache management. All DataCache tests pass. 2024-08-25 10:25:08 -03:00
Rob 8361efd965 Refactored DataCache, Database and Users. All db interactions are now all inside Database. All data request from Users now go through DataCache. Expanded DataCache test to include all methods. All DataCache tests pass. 2024-08-19 23:10:13 -03:00
Rob 439c852cf5 Fixed an issue where the client asked for too many records and was not receiving them. Also ensured that much of the code is dealing with timezones proactively. 2024-08-16 15:51:15 -03:00
Rob c398a423a3 implemented tests for Exchangeinterface.py 2024-08-04 02:06:23 -03:00
Rob 4130e0ca9a Completed unittests for Database and DataCache. 2024-08-03 16:56:13 -03:00
Rob 0b1ad39476 Fixed query_uptodate function and added comprehensive test cases 2024-08-02 01:36:45 -03:00
Rob b699bd950e Updated project structure and staged modified files 2024-07-25 08:21:15 -03:00