1. Research question
Can price, volume, volatility, momentum, and market-context features improve short-term directional classification of SPY beyond a naive majority-class baseline when evaluated with leakage-safe walk-forward methodology?
2. Data
Primary instrument: SPY daily OHLCV (Yahoo Finance). Context series: VIX, 10-year Treasury yield (^TNX), and S&P 500 (^GSPC). History spans 2015-01-02 → 2026-09-01 (~10+ years; early rows warm up features).
Target: direction_next_day — 1 if Close_(t+1) > Close_t else 0. Features at time t use only information known at or before that day's close.
3. Feature engineering
29 trailing features. No centered windows. Groups: returns, trend, momentum, volatility, volume, market context.
| Feature | Lookback | Intuition |
|---|---|---|
| ret_1d | 1d | Immediate price momentum / mean-reversion cue. |
| ret_2d | 2d | Very short-horizon cumulative return. |
| ret_5d | 5d | One-week return. |
| ret_10d | 10d | Two-week return. |
| ret_20d | 20d | Approx. one-month return. |
| close_sma5 | 5d | Price vs short moving average. |
| close_sma10 | 10d | Price vs 10-day MA. |
| close_sma20 | 20d | Price vs 20-day MA. |
| close_sma50 | 50d | Price vs medium-term MA. |
| sma5_sma20 | 20d | Short vs intermediate trend alignment. |
| sma20_sma50 | 50d | Intermediate vs longer trend alignment. |
| rsi_14 | 14d | Overbought/oversold momentum oscillator. |
| macd | 26d | Trend/momentum differential. |
| macd_signal_diff | 35d | MACD histogram / signal crossover strength. |
| mom_10 | 10d | Absolute 10-day price momentum. |
| vol_5d | 5d | Short-horizon realized volatility (ann.). |
| vol_10d | 10d | 10-day realized volatility (ann.). |
| vol_20d | 20d | 20-day realized volatility (ann.). |
| atr_14 | 14d | Normalized average true range. |
| hl_range | 1d | Intraday high-low range as % of close. |
| vol_pct_change | 1d | Day-over-day volume change. |
| vol_sma20_ratio | 20d | Volume relative to recent average. |
| vol_zscore_20 | 20d | Volume z-score vs 20-day history. |
| vix_level | 1d | Implied volatility / fear gauge level. |
| vix_change_1d | 1d | Daily change in implied volatility. |
| tnx_level | 1d | Treasury yield level (rate regime). |
| tnx_change_5d | 5d | Short-horizon yield move. |
| dist_from_high_20 | 20d | Distance from recent 20-day high. |
| dist_from_low_20 | 20d | Distance from recent 20-day low. |
Reference model feature importance
XGBoost split importance (association, not causality). Top features shown below.
| Feature | Importance |
|---|---|
| close_sma50 | 0.0396 |
| atr_14 | 0.0389 |
| vol_20d | 0.0389 |
| close_sma20 | 0.0388 |
| ret_20d | 0.0376 |
| tnx_change_5d | 0.0369 |
| tnx_level | 0.0367 |
| sma20_sma50 | 0.0365 |
| dist_from_high_20 | 0.0364 |
| mom_10 | 0.0358 |
| vol_10d | 0.0357 |
| ret_1d | 0.0353 |
4. Models
- Majority-class baseline
- Logistic regression (scaled, class-weighted)
- XGBoost classifier
- Small neural net (MLP, early stopping)
Selected for trading: XGBoost by validation roc_auc, then f1. Selection used only pre-holdout walk-forward results; simulation window not used for tuning.
| Model | Accuracy | Precision | Recall | F1 | ROC-AUC | vs Baseline |
|---|---|---|---|---|---|---|
| Majority Baseline | 54.79% | 54.79% | 100.00% | 0.708 | 0.470 | 0.00% |
| Logistic Regression | 51.81% | 57.69% | 45.19% | 0.507 | 0.514 | -2.98% |
| XGBoostSELECTED | 51.94% | 56.05% | 56.89% | 0.565 | 0.516 | -2.86% |
| Neural Net | 52.19% | 54.76% | 73.35% | 0.627 | 0.497 | -2.60% |
Confusion matrix (XGBoost): TN 327, FP 385, FN 372, TP 491.
5. Walk-forward validation
Random train/test splits are inappropriate for time series — they leak the future. This project uses an expanding training window and quarterly (~63-day) test folds. The final 500 trading days were held out from model selection.
Random train/test splitting would leak future information into training and overstate accuracy. Walk-forward evaluation respects time: every prediction is made without seeing the future.
During the simulation, models were retrained on a walk-forward schedule: every 21 trading days (approximately monthly).
6. Trading rules & execution
- If P(up) ≥ 0.5 → hold SPY; else cash
- Signal after close t → execute at open t+1 (signal after close(t) → execute open(t+1))
- Costs: 5 bps transaction + 2 bps slippage per side
- Starting capital $1,000,000; no shorting or leverage
- Window: Sep 4, 2024 → Sep 1, 2026 (500 days)
Trade ledger
All 127 fills from the simulation, paired into round-trips with entry/exit, hold time, and P&L after costs.
| # | Entry | Exit | Hold | Entry $ | Exit $ | P&L | Return | P(up) | Status |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Sep 5, 2024 | Sep 9, 2024 | 4d | $551.28 | $544.27 | -$12,710 | -1.27% | 0.503 | Closed |
| 2 | Sep 10, 2024 | Oct 8, 2024 | 28d | $548.74 | $570.02 | $38,281 | +3.88% | 0.540 | Closed |
| 3 | Oct 10, 2024 | Oct 15, 2024 | 5d | $576.17 | $584.18 | $14,254 | +1.39% | 0.547 | Closed |
| 4 | Oct 16, 2024 | Oct 24, 2024 | 8d | $580.19 | $579.57 | -$1,097 | -0.11% | 0.615 | Closed |
| 5 | Oct 25, 2024 | Nov 1, 2024 | 7d | $581.92 | $570.92 | -$19,630 | -1.89% | 0.604 | Closed |
| 6 | Nov 4, 2024 | Nov 7, 2024 | 3d | $571.58 | $592.66 | $37,594 | +3.69% | 0.569 | Closed |
| 7 | Nov 8, 2024 | Nov 18, 2024 | 10d | $596.59 | $585.81 | -$19,090 | -1.81% | 0.674 | Closed |
| 8 | Nov 19, 2024 | Dec 19, 2024 | 30d | $585.12 | $590.95 | $10,333 | +1.00% | 0.617 | Closed |
| 9 | Dec 27, 2024 | Jan 6, 2025 | 10d | $597.96 | $595.85 | -$3,690 | -0.35% | 0.590 | Closed |
| 10 | Jan 7, 2025 | Jan 8, 2025 | 1d | $597.84 | $588.29 | -$16,681 | -1.60% | 0.528 | Closed |
| 11 | Jan 10, 2025 | Jan 13, 2025 | 3d | $586.29 | $575.37 | -$19,144 | -1.86% | 0.537 | Closed |
| 12 | Jan 16, 2025 | Jan 17, 2025 | 1d | $594.59 | $596.54 | $3,318 | +0.33% | 0.574 | Closed |
| 13 | Jan 21, 2025 | Feb 19, 2025 | 29d | $601.09 | $609.65 | $14,412 | +1.42% | 0.714 | Closed |
| 14 | Feb 20, 2025 | Mar 31, 2025 | 39d | $611.97 | $549.45 | -$104,838 | -10.22% | 0.626 | Closed |
| 15 | Apr 1, 2025 | Apr 7, 2025 | 6d | $557.84 | $488.85 | -$113,946 | -12.37% | 0.568 | Closed |
| 16 | Apr 8, 2025 | Apr 9, 2025 | 1d | $522.23 | $493.09 | -$45,036 | -5.58% | 0.530 | Closed |
| 17 | Apr 10, 2025 | Apr 11, 2025 | 1d | $532.54 | $522.64 | -$14,170 | -1.86% | 0.519 | Closed |
| 18 | Apr 17, 2025 | Apr 25, 2025 | 8d | $528.01 | $546.27 | $25,871 | +3.46% | 0.581 | Closed |
| 19 | Apr 28, 2025 | May 2, 2025 | 4d | $551.78 | $564.33 | $17,617 | +2.28% | 0.516 | Closed |
| 20 | May 6, 2025 | May 13, 2025 | 7d | $558.32 | $583.00 | $34,995 | +4.42% | 0.555 | Closed |
| 21 | May 27, 2025 | May 28, 2025 | 1d | $586.48 | $591.15 | $6,576 | +0.80% | 0.651 | Closed |
| 22 | May 30, 2025 | Jun 2, 2025 | 3d | $589.34 | $587.35 | -$2,819 | -0.34% | 0.514 | Closed |
| 23 | Jun 12, 2025 | Jun 13, 2025 | 1d | $600.43 | $598.08 | -$3,249 | -0.39% | 0.516 | Closed |
| 24 | Jun 23, 2025 | Jun 26, 2025 | 3d | $595.46 | $608.56 | $18,207 | +2.20% | 0.588 | Closed |
| 25 | Jun 27, 2025 | Jul 1, 2025 | 4d | $613.31 | $615.93 | $3,611 | +0.43% | 0.607 | Closed |
| 26 | Jul 2, 2025 | Jul 7, 2025 | 5d | $617.67 | $622.92 | $7,218 | +0.85% | 0.559 | Closed |
| 27 | Jul 8, 2025 | Jul 11, 2025 | 3d | $621.78 | $622.30 | $715 | +0.08% | 0.573 | Closed |
| 28 | Jul 16, 2025 | Jul 18, 2025 | 2d | $624.18 | $628.86 | $6,429 | +0.75% | 0.645 | Closed |
| 29 | Jul 21, 2025 | Jul 29, 2025 | 8d | $629.21 | $637.90 | $11,927 | +1.38% | 0.559 | Closed |
| 30 | Jul 30, 2025 | Aug 1, 2025 | 2d | $636.37 | $625.86 | -$14,447 | -1.65% | 0.530 | Closed |
| 31 | Aug 5, 2025 | Aug 19, 2025 | 14d | $632.23 | $642.67 | $14,211 | +1.65% | 0.675 | Closed |
| 32 | Aug 20, 2025 | Aug 28, 2025 | 8d | $639.85 | $646.79 | $9,490 | +1.08% | 0.562 | Closed |
| 33 | Aug 29, 2025 | Sep 16, 2025 | 18d | $647.92 | $661.01 | $17,861 | +2.02% | 0.568 | Closed |
| 34 | Sep 24, 2025 | Sep 26, 2025 | 2d | $664.98 | $659.05 | -$8,043 | -0.89% | 0.502 | Closed |
| 35 | Sep 29, 2025 | Oct 7, 2025 | 8d | $664.83 | $672.07 | $9,745 | +1.09% | 0.559 | Closed |
| 36 | Oct 9, 2025 | Oct 13, 2025 | 4d | $674.00 | $660.19 | -$18,529 | -2.05% | 0.509 | Closed |
| 37 | Oct 14, 2025 | Nov 3, 2025 | 20d | $657.63 | $685.19 | $37,112 | +4.19% | 0.513 | Closed |
| 38 | Nov 4, 2025 | Nov 14, 2025 | 10d | $676.58 | $664.91 | -$15,913 | -1.72% | 0.528 | Closed |
| 39 | Nov 18, 2025 | Nov 21, 2025 | 3d | $662.56 | $654.59 | -$10,910 | -1.20% | 0.505 | Closed |
| 40 | Nov 24, 2025 | Nov 25, 2025 | 1d | $663.15 | $668.16 | $6,765 | +0.76% | 0.562 | Closed |
| 41 | Nov 26, 2025 | Dec 4, 2025 | 8d | $678.10 | $684.82 | $8,939 | +0.99% | 0.558 | Closed |
| 42 | Dec 5, 2025 | Dec 11, 2025 | 6d | $685.95 | $684.66 | -$1,713 | -0.19% | 0.503 | Closed |
| 43 | Dec 15, 2025 | Dec 29, 2025 | 14d | $686.22 | $687.06 | $1,112 | +0.12% | 0.550 | Closed |
| 44 | Dec 30, 2025 | Jan 5, 2026 | 6d | $687.93 | $686.06 | -$2,479 | -0.27% | 0.520 | Closed |
| 45 | Jan 8, 2026 | Jan 13, 2026 | 5d | $689.30 | $695.00 | $7,514 | +0.83% | 0.547 | Closed |
| 46 | Jan 14, 2026 | Feb 9, 2026 | 26d | $691.48 | $688.94 | -$3,373 | -0.37% | 0.521 | Closed |
| 47 | Feb 10, 2026 | Feb 11, 2026 | 1d | $695.44 | $695.90 | $612 | +0.07% | 0.576 | Closed |
| 48 | Feb 12, 2026 | Feb 26, 2026 | 14d | $694.73 | $692.79 | -$2,539 | -0.28% | 0.554 | Closed |
| 49 | Feb 27, 2026 | Mar 3, 2026 | 4d | $683.57 | $674.59 | -$11,964 | -1.31% | 0.506 | Closed |
| 50 | Mar 4, 2026 | Mar 9, 2026 | 5d | $682.11 | $665.92 | -$21,323 | -2.37% | 0.547 | Closed |
| 51 | Mar 10, 2026 | Mar 23, 2026 | 13d | $678.19 | $657.61 | -$26,631 | -3.04% | 0.531 | Closed |
| 52 | Mar 24, 2026 | Apr 1, 2026 | 8d | $651.78 | $653.44 | $2,175 | +0.26% | 0.532 | Closed |
| 53 | Apr 2, 2026 | Apr 21, 2026 | 19d | $646.87 | $709.78 | $82,950 | +9.73% | 0.617 | Closed |
| 54 | Apr 22, 2026 | Apr 23, 2026 | 1d | $709.65 | $709.00 | -$848 | -0.09% | 0.561 | Closed |
| 55 | Apr 29, 2026 | Apr 30, 2026 | 1d | $711.50 | $714.13 | $3,459 | +0.37% | 0.536 | Closed |
| 56 | May 5, 2026 | May 11, 2026 | 6d | $722.28 | $735.93 | $17,748 | +1.89% | 0.535 | Closed |
| 57 | May 13, 2026 | May 14, 2026 | 1d | $738.99 | $743.13 | $5,360 | +0.56% | 0.584 | Closed |
| 58 | May 18, 2026 | May 27, 2026 | 9d | $740.35 | $750.35 | $12,997 | +1.35% | 0.617 | Closed |
| 59 | May 28, 2026 | May 29, 2026 | 1d | $750.78 | $755.37 | $5,966 | +0.61% | 0.594 | Closed |
| 60 | Jun 1, 2026 | Jun 3, 2026 | 2d | $755.89 | $757.62 | $2,245 | +0.23% | 0.561 | Closed |
| 61 | Jun 4, 2026 | Jun 5, 2026 | 1d | $752.63 | $751.78 | -$1,101 | -0.11% | 0.522 | Closed |
| 62 | Jun 8, 2026 | Jun 10, 2026 | 2d | $743.88 | $732.88 | -$14,522 | -1.48% | 0.505 | Closed |
| 63 | Jun 11, 2026 | Jul 31, 2026 | 50d | $729.27 | $744.16 | $19,746 | +2.04% | 0.599 | Closed |
| 64 | Aug 4, 2026 | — | — | $761.16 | — | — | — | 0.599 | Open |
Round-trips pair each SELL with the prior unclosed BUY. Prices include transaction costs and slippage. P&L is mark-to-fill for closed positions only.
7. Results
| Metric | ML strategy | SPY buy & hold |
|---|---|---|
| Ending value | $987,730 | $1,383,582 |
| Total return | -1.23% | +38.27% |
| Sharpe | 0.026 | 1.068 |
| Max drawdown | -32.44% | -19.00% |
| Ann. volatility | 14.0% | 16.6% |
| Exposure | 76.8% | 100% |
Simulation directional accuracy: 52.2%. Relative return vs benchmark: -39.49%.
8. Limitations & lessons
- Daily equity direction is noisy; majority baselines are strong in uptrends.
- Small accuracy differences can disappear after costs and next-open execution.
- Leakage prevention and chronology matter more than flashy accuracy claims.
- Yahoo Finance is convenient, not institutional-grade.
- An attractive backtest does not prove future profitability.