AI Trading Platform: Observing Model Behavior During Strategy Optimization

Initial Observations

In our last post, we built an evaluation framework for measuring system behavior. With that in place, we could observe how the model performed under test conditions.

Initial metrics showed:

A Profit Factor below 1.0 indicates negative expected value under the current configuration.

Research Objectives

We defined observation targets for iterative testing:

1. Profit Factor > 1.0 — Positive expected value
2. Win Rate stability — Consistent outcome distribution
3. Loss magnitude reduction — Smaller drawdowns per trade
4. Risk/Reward ratio — Favorable return distribution

Iteration Log: 15 Versions

The following table documents observed behavior changes across iterations:

Non-monotonic behavior was observed. Parameter changes did not consistently improve metrics.

Observed Constraints

Observation 1: Stop-Loss Distance

The initial stop-loss was set at 2.0 ATR (Average True Range). This allowed significant adverse movement before exit. Average loss per stopped trade was -253%.

This represented the largest contributor to negative expected value.

Observation 2: Time Period Sensitivity

Performance varied significantly by time period:

Certain time horizons showed consistent negative outcomes. This pattern was not anticipated.

Observation 3: Symbol-Specific Behavior

Certain symbols showed divergent behavior:

Model performance was not uniform across all instruments.

Observation 4: Breakeven Stop Behavior

A feature moved the stop loss to breakeven after small gains. In theory this reduces risk. In practice, trading costs (fees, spread, slippage) meant breakeven exits produced net negative returns.

This feature contributed to loss accumulation rather than risk reduction.

Parameter Adjustments

The following changes were tested:

Adjustment 1: Stop-Loss Distance

ATR multiplier reduced from 2.0 to 0.6:

Observed effect: Average stopped loss changed from -253% to -0.81%. This represented the largest shift in loss magnitude.

Adjustment 2: Trailing Stop Timing

Trailing stop activation was adjusted:

This configuration triggered profit protection earlier in the trade lifecycle.

Adjustment 3: Time Period Exclusion

Time periods with observed negative performance were excluded:

Excluded: 1 week, 1 month, 2 months, 3 months, 6 months, 1 year

The system no longer generated signals for these time horizons.

Adjustment 4: Symbol Exclusion

Symbols with divergent behavior were excluded:

Excluded: META

Adjustment 5: Breakeven Stop Disabled

The breakeven stop feature was disabled. Its contribution to loss accumulation outweighed its intended risk reduction.

Observed Metric Changes

The following table shows metric distribution before and after parameter adjustments:

These observations reflect one configuration state. Generalizability has not been established.

Sample Distribution

The test sample included 171 trades:

Research Notes

Observations from this iteration cycle:

1. Stop-loss distance affects loss magnitude

Wider stops allowed larger adverse movements before exit. Tighter stops reduced average loss per trade.

2. Time period sensitivity exists

Certain time horizons showed consistent negative outcomes. Segmenting analysis by time period revealed patterns not visible in aggregate data.

3. Earlier trailing affects return capture

Delayed trailing allowed gains to reverse. Earlier activation reduced this effect.

4. Feature interaction with costs

The breakeven stop appeared beneficial in isolation. When trading costs were included, it contributed to losses. Features should be tested with realistic cost models.

5. Non-monotonic optimization

15 iterations were required to observe significant metric shifts. Intermediate versions showed regression. Sequential iteration was necessary.

Configuration Snapshot

The following configuration reflects one observed state of the system during testing. It is not presented as an optimal or transferable setup.

Next Research Phases

The initial observation target (Profit Factor > 1.0) was observed in this configuration.

Further stages will focus on:

1. Out-of-sample testing — Observing behavior on unseen data
2. Live data observation — Monitoring system responses to real-time data streams
3. Stability analysis — Measuring metric variance over extended periods

No conclusions about deployability are drawn from these observations.

The largest observed effect came from a single parameter change: reducing the ATR multiplier from 2.0 to 0.6 shifted average loss from -253% to -0.81%. This suggests stop-loss distance may be a high-leverage variable in similar configurations.

Learn more about the platform: AI Trading Platform

Previous: Building a Bulletproof Evaluation Framework

This document records observations from iterative testing. Results are specific to the tested configuration and data set.