Phase 0: Stabilize baseline

- Security: Move API keys to environment variables in config.py - Portability: Use cross-platform path resolution for DB_FILE - Add config.example.py template for developers - Fix Windows path in ExchangeInterface.get_public_exchanges() - Add cached_last_candle attribute to Candles class - Add pytest configuration (pytest.ini, conftest.py) - Fix test imports for DataCache_v3 - Include identity compatibility layer (user_name/user_id resolution) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 16:44:11 -04:00 · 2026-02-28 16:44:11 -04:00 · 34637df478
parent f1a184b131
commit 34637df478
11 changed files with 401 additions and 28 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@ -0,0 +1,130 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+**Brighter Trading** - Cryptocurrency trading platform with Blockly-based strategy builder
+
+## ⚠️ CRITICAL: Updating Todos, Milestones, and Goals
+
+**DO NOT edit `todos.md`, `milestones.md`, or `goals.md` files directly.**
+
+These files are managed by Development Hub which has file watchers and sync logic. Direct edits will be overwritten or cause conflicts.
+
+**Use the `devhub-tasks` CLI instead:**
+
+```bash
+# Status overview
+devhub-tasks status brightertrading
+
+# Add todos
+devhub-tasks todo add brightertrading "Task description" --priority high --milestone M1
+
+# Complete todos (by text match or ID number)
+devhub-tasks todo complete brightertrading "Task description"
+devhub-tasks todo complete brightertrading 3
+
+# List todos
+devhub-tasks todo list brightertrading
+
+# Add milestones
+devhub-tasks milestone add brightertrading M2 --name "Milestone Name" --target "March 2026"
+
+# Complete milestones (also completes linked todos)
+devhub-tasks milestone complete brightertrading M1
+
+# Goals
+devhub-tasks goal add brightertrading "Goal description" --priority high
+devhub-tasks goal complete brightertrading "Goal description"
+```
+
+Use `--json` flag for machine-readable output. Run `devhub-tasks --help` for full documentation.
+
+**Files you CAN edit directly:** `overview.md`, `architecture.md`, `README.md`, and any other docs.
+
+## Development Commands
+
+```bash
+# Install for development
+pip install -e ".[dev]"
+
+# Run tests
+pytest
+
+# Run a single test
+pytest tests/test_file.py::test_name
+```
+
+## Architecture
+
+Flask web application with SocketIO for real-time communication, using eventlet for async operations. Features a Blockly-based visual strategy builder frontend.
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                        Flask + SocketIO                         │
+│                          (app.py)                               │
+├─────────────────────────────────────────────────────────────────┤
+│                      BrighterTrades                             │
+│              (Main application facade/coordinator)              │
+├─────────┬─────────┬─────────┬─────────┬─────────┬──────────────┤
+│  Users  │Strategies│ Trades │Indicators│Backtester│ Exchanges   │
+├─────────┴─────────┴─────────┴─────────┴─────────┴──────────────┤
+│                         DataCache                               │
+│               (In-memory caching + SQLite)                      │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+### Key Modules
+
+| Module | Purpose |
+|--------|---------|
+| `app.py` | Flask web server, SocketIO handlers, HTTP routes |
+| `BrighterTrades.py` | Main application facade, coordinates all subsystems |
+| `Strategies.py` | Strategy CRUD operations and execution management |
+| `StrategyInstance.py` | Individual strategy execution context |
+| `PythonGenerator.py` | Generates Python code from Blockly JSON |
+| `backtesting.py` | Strategy backtesting engine (uses backtrader) |
+| `indicators.py` | Technical indicator calculations |
+| `candles.py` | Candlestick/OHLCV data management |
+| `trade.py` | Trade lifecycle management |
+| `ExchangeInterface.py` | Multi-exchange interface |
+| `Exchange.py` | Single exchange wrapper (uses ccxt) |
+| `DataCache_v3.py` | In-memory caching with database persistence |
+| `Database.py` | SQLite database operations |
+| `Users.py` | User authentication and session management |
+| `Signals.py` | Trading signal definitions and state tracking |
+
+### Key Paths
+
+- **Source code**: `src/`
+- **Tests**: `tests/`
+- **Configuration**: `config.yml`
+- **Database**: `BrighterTrading.db` (SQLite)
+- **Documentation**: `docs/` (symlink to project-docs)
+
+### Running the Application
+
+```bash
+# Start the development server
+cd src && python app.py
+
+# Or from project root
+python src/app.py
+```
+
+The application runs on `http://127.0.0.1:5002` by default.
+
+## Documentation
+
+Documentation lives in `docs/` (symlink to centralized docs system).
+
+**Before updating docs, read `docs/updating-documentation.md`** for full details on visibility rules and procedures.
+
+Quick reference:
+- Edit files in `docs/` folder
+- Use `public: true` frontmatter for public-facing docs
+- Use `<!-- PRIVATE_START -->` / `<!-- PRIVATE_END -->` to hide sections
+- Deploy: `~/PycharmProjects/project-docs/scripts/build-public-docs.sh brightertrading --deploy`
+
+Do NOT create documentation files directly in this repository.
--- a/pytest.ini
+++ b/pytest.ini
@ -0,0 +1,6 @@
+[pytest]
+testpaths = tests
+python_files = test_*.py
+python_classes = Test*
+python_functions = test_*
+addopts = -v --tb=short
--- a/src/BrighterTrades.py
+++ b/src/BrighterTrades.py
@ -56,6 +56,53 @@ class BrighterTrades:
                                     indicators=self.indicators, socketio=socketio)
        self.backtests = {}  # In-memory storage for backtests (replace with DB access in production)

+    @staticmethod
+    def _coerce_user_id(user_id: Any) -> int | None:
+        if user_id is None or user_id == '':
+            return None
+        try:
+            return int(user_id)
+        except (TypeError, ValueError):
+            return None
+
+    def resolve_user_name(self, msg_data: dict | None) -> str | None:
+        """
+        Resolve a username from payload fields, accepting both legacy and migrated key shapes.
+        """
+        if not isinstance(msg_data, dict):
+            return None
+
+        user_name = msg_data.get('user_name') or msg_data.get('user')
+        if user_name:
+            return user_name
+
+        user_id = self._coerce_user_id(msg_data.get('user_id') or msg_data.get('userId'))
+        if user_id is None:
+            return None
+
+        try:
+            return self.users.get_username(user_id=user_id)
+        except Exception:
+            logger.warning(f"Unable to resolve user_name from user id '{user_id}'.")
+            return None
+
+    def resolve_user_id(self, msg_data: dict | None, user_name: str | None = None) -> int | None:
+        """
+        Resolve a user id from payload fields, accepting both legacy and migrated key shapes.
+        """
+        if isinstance(msg_data, dict):
+            user_id = self._coerce_user_id(msg_data.get('user_id') or msg_data.get('userId'))
+            if user_id is not None:
+                return user_id
+
+        if user_name:
+            try:
+                return self.get_user_info(user_name=user_name, info='User_id')
+            except Exception:
+                logger.warning(f"Unable to resolve user_id from user_name '{user_name}'.")
+                return None
+        return None
+
    def create_new_user(self, email: str, username: str, password: str) -> bool:
        """
            Creates a new user and logs the user in.
@ -645,7 +692,7 @@ class BrighterTrades:
    def delete_backtest(self, msg_data):
        """ Delete an existing backtest by interacting with the Backtester. """
        backtest_name = msg_data.get('name')
-        user_name = msg_data.get('user_name')
+        user_name = self.resolve_user_name(msg_data)

        if not backtest_name or not user_name:
            return {"success": False, "message": "Missing backtest name or user name."}
@ -751,6 +798,16 @@ class BrighterTrades:
            """ Formats a standard reply message. """
            return {"reply": reply_msg, "data": reply_data}

+        user_name = self.resolve_user_name(msg_data)
+        user_id = self.resolve_user_id(msg_data, user_name=user_name)
+
+        if user_name:
+            msg_data.setdefault('user_name', user_name)
+            msg_data.setdefault('user', user_name)
+        if user_id is not None:
+            msg_data.setdefault('user_id', user_id)
+            msg_data.setdefault('userId', user_id)
+
        if msg_type == 'candle_data':
            if r_data := self.received_cdata(msg_data):
                return standard_reply("updates", r_data)
@ -762,7 +819,8 @@ class BrighterTrades:
                    return standard_reply("signals", signals)

            elif request_for == 'strategies':
-                user_id = self.get_user_info(msg_data['user_name'], 'User_id')
+                if user_id is None:
+                    return standard_reply("strategy_error", {"message": "User not specified"})
                if strategies := self.get_strategies_json(user_id):
                    return standard_reply("strategies", strategies)

@ -820,21 +878,31 @@ class BrighterTrades:
                return standard_reply("trade_created", r_data)

        if msg_type == 'config_exchange':
-            user, exchange, keys = msg_data['user'], msg_data['exch'], msg_data['keys']
+            user = msg_data.get('user') or user_name
+            exchange = msg_data.get('exch') or msg_data.get('exchange') or msg_data.get('exchange_name')
+            keys = msg_data.get('keys')
+            if not user or not exchange:
+                return standard_reply("Exchange_connection_result", {
+                    "exchange": exchange or '',
+                    "status": "error",
+                    "message": "Missing user or exchange in request."
+                })
            r_data = self.connect_or_config_exchange(user_name=user, exchange_name=exchange, api_keys=keys)
            return standard_reply("Exchange_connection_result", r_data)

        # Handle backtest operations
        if msg_type == 'submit_backtest':
            # Validate required fields
-            required_fields = ['strategy', 'start_date', 'capital', 'commission', 'user_name']
+            required_fields = ['strategy', 'start_date', 'capital', 'commission']
            if not all(field in msg_data for field in required_fields):
                return standard_reply("backtest_error", {"message": "Missing required fields."})
+            if not user_name:
+                return standard_reply("backtest_error", {"message": "Missing user identity."})

            try:
                # Delegate backtest handling to the Backtester
                resp = self.backtester.handle_backtest_message(
-                    user_id=self.get_user_info(user_name=msg_data['user_name'], info='User_id'),
+                    user_id=user_id if user_id is not None else self.get_user_info(user_name=user_name, info='User_id'),
                    msg_data=msg_data,
                    socket_conn_id=socket_conn_id
                )
--- a/src/ExchangeInterface.py
+++ b/src/ExchangeInterface.py
@ -85,8 +85,11 @@ class ExchangeInterface:
    @staticmethod
    def get_public_exchanges() -> List[str]:
        """Return a list of public exchanges available from CCXT."""
+        from pathlib import Path
+
        public_list = []
-        file_path = r"src\working_public_exchanges.txt"
+        # Cross-platform path resolution
+        file_path = Path(__file__).parent / 'working_public_exchanges.txt'

        try:
            with open(file_path, 'r') as file:
--- a/src/app.py
+++ b/src/app.py
@ -57,6 +57,37 @@ def add_cors_headers(response):
    return response


+def _coerce_user_id(user_id):
+    if user_id is None or user_id == '':
+        return None
+    try:
+        return int(user_id)
+    except (TypeError, ValueError):
+        return None
+
+
+def resolve_user_name(payload: dict | None) -> str | None:
+    """
+    Resolve a username from payload fields, accepting both legacy and migrated key shapes.
+    """
+    if not isinstance(payload, dict):
+        return None
+
+    user_name = payload.get('user_name') or payload.get('user')
+    if user_name:
+        return user_name
+
+    user_id = _coerce_user_id(payload.get('user_id') or payload.get('userId'))
+    if user_id is None:
+        return None
+
+    try:
+        return brighter_trades.users.get_username(user_id=user_id)
+    except Exception:
+        logging.warning(f"Unable to resolve user_name from user id '{user_id}'.")
+        return None
+
+
@app.route('/')
 # @cross_origin(supports_credentials=True)
 def index():
@ -119,6 +150,11 @@ def index():
@socketio.on('connect')
 def handle_connect():
    user_name = request.args.get('user_name')
+    if not user_name:
+        user_name = resolve_user_name({
+            'userId': request.args.get('userId'),
+            'user_id': request.args.get('user_id')
+        })
    if user_name and brighter_trades.get_user_info(user_name=user_name, info='Is logged in?'):
        # Join a room specific to the user for targeted messaging
        room = user_name  # You can choose an appropriate room naming strategy
@ -143,11 +179,20 @@ def handle_message(data):
    msg_type, msg_data = data['message_type'], data['data']

    # Extract user_name from the incoming message data
-    user_name = msg_data.get('user_name')
+    user_name = resolve_user_name(msg_data)
    if not user_name:
        emit('message', {"success": False, "message": "User not specified"})
        return

+    msg_data.setdefault('user_name', user_name)
+    try:
+        user_id = brighter_trades.get_user_info(user_name=user_name, info='User_id')
+        if user_id is not None:
+            msg_data.setdefault('user_id', user_id)
+            msg_data.setdefault('userId', user_id)
+    except Exception:
+        pass
+
    # Check if the user is logged in
    if not brighter_trades.get_user_info(user_name=user_name, info='Is logged in?'):
        emit('message', {"success": False, "message": "User not logged in"})
@ -226,7 +271,7 @@ def history():
        if not data:
            raise ValueError("No JSON data received")

-        username = data.get('user_name')
+        username = resolve_user_name(data)

        # Return if the user is not logged in.
        if not username:
@ -325,7 +370,7 @@ def indicator_init():
    Initializes the indicators and returns the data for a given symbol and timeframe.
    """
    data = request.get_json()
-    username = data.get('user_name')
+    username = resolve_user_name(data)

    if not username:
        return jsonify({'error': 'Invalid user name.'}), 400
--- a/src/backtesting.py
+++ b/src/backtesting.py
@ -71,6 +71,55 @@ class Backtester:
        signal.signal(signal.SIGINT, self.shutdown_handler)
        signal.signal(signal.SIGTERM, self.shutdown_handler)

+    @staticmethod
+    def _coerce_user_id(user_id) -> int | None:
+        if user_id is None or user_id == '':
+            return None
+        try:
+            return int(user_id)
+        except (TypeError, ValueError):
+            return None
+
+    def _resolve_user_name(self, msg_data: dict) -> str | None:
+        user_name = msg_data.get('user_name') or msg_data.get('user')
+        if user_name:
+            return user_name
+
+        user_id = self._coerce_user_id(msg_data.get('user_id') or msg_data.get('userId'))
+        if user_id is None:
+            return None
+
+        try:
+            return self.data_cache.get_datacache_item(
+                item_name='user_name',
+                cache_name='users',
+                filter_vals=('id', user_id)
+            )
+        except Exception:
+            logger.warning(f"Unable to resolve user_name from user id '{user_id}'.")
+            return None
+
+    def _resolve_user_id(self, msg_data: dict, user_id=None, user_name: str | None = None) -> int | None:
+        normalized_user_id = self._coerce_user_id(user_id)
+        if normalized_user_id is not None:
+            return normalized_user_id
+
+        normalized_user_id = self._coerce_user_id(msg_data.get('user_id') or msg_data.get('userId'))
+        if normalized_user_id is not None:
+            return normalized_user_id
+
+        if user_name:
+            try:
+                return self.data_cache.get_datacache_item(
+                    item_name='id',
+                    cache_name='users',
+                    filter_vals=('user_name', user_name)
+                )
+            except Exception:
+                logger.warning(f"Unable to resolve user_id from user_name '{user_name}'.")
+                return None
+        return None
+
    def cache_backtest(self, backtest_key: str, backtest_data: dict, strategy_instance_id: str):
        """
        Cache the backtest data for a user.
@ -332,7 +381,9 @@ class Backtester:
        :return: Tuple of (data_feed, precomputed_indicators).
        :raises ValueError: If data sources are invalid or data feed cannot be prepared.
        """
-        user_name = msg_data.get('user_name', 'default_user')
+        user_name = self._resolve_user_name(msg_data)
+        if not user_name:
+            raise ValueError("User not specified in backtest request.")

        data_sources = strategy_components.get('data_sources', [])

@ -478,7 +529,15 @@ class Backtester:
        Handle incoming backtest messages, orchestrate the backtest process.
        """
        # Extract and define backtest parameters
-        user_name = msg_data.get('user_name')
+        user_name = self._resolve_user_name(msg_data)
+        user_id = self._resolve_user_id(msg_data, user_id=user_id, user_name=user_name)
+        if not user_name or user_id is None:
+            return {"error": "Missing user identity for backtest request."}
+
+        msg_data.setdefault('user_name', user_name)
+        msg_data.setdefault('user_id', user_id)
+        msg_data.setdefault('userId', user_id)
+
        tbl_key = msg_data.get('strategy')  # Expecting tbl_key instead of strategy_name
        backtest_name = msg_data.get('backtest_name')  # Use the client-provided backtest_name

@ -512,7 +571,7 @@ class Backtester:
            strategy_instance_id=strategy_instance_id,
            strategy_id=tbl_key,  # Use tbl_key as strategy_id
            strategy_name=user_strategy.get("name"),
-            user_id=int(user_id),
+            user_id=user_id,
            generated_code=strategy_components.get("generated_code", ""),
            data_cache=self.data_cache,
            indicators=None,  # Custom handling in BacktestStrategyInstance
@ -879,4 +938,4 @@ class Backtester:
            ret = (equity_curve[i] - equity_curve[i - 1]) / equity_curve[i - 1]
            returns.append(ret)
        logger.debug(f"Calculated returns: {returns}")
-        return returns
+        return returns
--- a/src/candles.py
+++ b/src/candles.py
@ -17,6 +17,9 @@ class Candles:
        # This object maintains all the cached data.
        self.data = datacache

+        # Cache the last received candle to detect duplicates
+        self.cached_last_candle = None
+
        # size_limit is the max number of lists of candle(ohlc) data allowed.
        self.data.create_cache(name='candles', cache_type='row', default_expiration=dt.timedelta(days=5),
                               size_limit=100, eviction_policy='evict')
--- a/src/config.example.py
+++ b/src/config.example.py
@ -0,0 +1,31 @@
+"""
+Configuration module for BrighterTrading - EXAMPLE TEMPLATE.
+
+Copy this file to config.py and either:
+1. Set the environment variables below, OR
+2. Directly assign your keys (NOT recommended for production)
+
+Environment variables:
+  BRIGHTER_BINANCE_API_KEY
+  BRIGHTER_BINANCE_API_SECRET
+  BRIGHTER_ALPACA_API_KEY
+  BRIGHTER_ALPACA_API_SECRET
+"""
+import os
+from pathlib import Path
+
+# API Keys - loaded from environment variables
+# To set environment variables:
+#   Linux/Mac: export BRIGHTER_BINANCE_API_KEY="your_key_here"
+#   Windows:   set BRIGHTER_BINANCE_API_KEY=your_key_here
+BINANCE_API_KEY = os.environ.get('BRIGHTER_BINANCE_API_KEY', '')
+BINANCE_API_SECRET = os.environ.get('BRIGHTER_BINANCE_API_SECRET', '')
+ALPACA_API_KEY = os.environ.get('BRIGHTER_ALPACA_API_KEY', '')
+ALPACA_API_SECRET = os.environ.get('BRIGHTER_ALPACA_API_SECRET', '')
+
+# Database path - cross-platform, relative to project root
+_project_root = Path(__file__).parent.parent
+DB_FILE = str(_project_root / 'data' / 'BrighterTrading.db')
+
+# Ensure data directory exists
+os.makedirs(os.path.dirname(DB_FILE), exist_ok=True)
--- a/tests/conftest.py
+++ b/tests/conftest.py
@ -0,0 +1,18 @@
+"""
+Pytest configuration for BrighterTrading tests.
+
+This module sets up the Python path to include the src directory,
+allowing tests to import modules with or without the 'src.' prefix.
+"""
+import sys
+from pathlib import Path
+
+# Add project root to Python path (for 'from src.X import Y' style)
+project_root = Path(__file__).parent.parent
+if str(project_root) not in sys.path:
+    sys.path.insert(0, str(project_root))
+
+# Add src directory to Python path (for 'from X import Y' style)
+src_path = project_root / 'src'
+if str(src_path) not in sys.path:
+    sys.path.insert(0, str(src_path))
--- a/tests/test_Users.py
+++ b/tests/test_Users.py
@ -1,17 +1,18 @@
 import json
+import pytest

 from Configuration import Configuration
-from DataCache import DataCache
+from DataCache_v3 import DataCache
 from ExchangeInterface import ExchangeInterface

-# Object that interacts and maintains exchange_interface and account data
-exchanges = ExchangeInterface()
-
 # Object that interacts with the persistent data.
-data = DataCache(exchanges)
+data = DataCache()
+
+# Object that interacts and maintains exchange_interface and account data
+exchanges = ExchangeInterface(data)

 # Configuration and settings for the user app and charts
-config = Configuration(cache=data)
+config = Configuration()


 def test_get_indicators():
--- a/tests/test_indicators.py
+++ b/tests/test_indicators.py
@ -1,29 +1,38 @@
 import json
+import os
+import pytest

 from Configuration import Configuration
-from DataCache import DataCache
+from DataCache_v3 import DataCache
 from candles import Candles
 from ExchangeInterface import ExchangeInterface
 from indicators import Indicators
+from Users import Users

-ALPACA_API_KEY = 'PKPSH7OHWH3Q5AUBZBE5'
-ALPACA_API_SECRET = 'dVy0AgQBgGV52cpwenTqAQcr1IGj7whkEEjPY6HB'
+# API keys from environment variables
+ALPACA_API_KEY = os.environ.get('BRIGHTER_ALPACA_API_KEY', '')
+ALPACA_API_SECRET = os.environ.get('BRIGHTER_ALPACA_API_SECRET', '')


+@pytest.mark.skipif(not ALPACA_API_KEY, reason="ALPACA API keys not configured")
 def test_indicators():
-    # Object that interacts and maintains exchange_interface and account data
-    exchanges = ExchangeInterface()
    # Object that interacts with the persistent data.
-    data = DataCache(exchanges)
+    data = DataCache()
+
+    # Object that interacts and maintains exchange_interface and account data
+    exchanges = ExchangeInterface(data)

    # Configuration and settings for the user app and charts
-    config = Configuration(cache=data)
+    config = Configuration()
+
+    # Object that manages users in the system.
+    users = Users(data_cache=data)

    # Object that maintains candlestick and price data.
-    candles = Candles(config_obj=config, exchanges=exchanges, database=data)
+    candles = Candles(exchanges=exchanges, users=users, datacache=data, config=config)

    # Object that interacts with and maintains data from available indicators
-    ind_obj = Indicators(candles, config)
+    ind_obj = Indicators(candles, users, data)

    user_name = 'guest'
    exchanges.connect_exchange('alpaca', user_name=user_name, api_keys={'key': ALPACA_API_KEY,