Core Engineering Takeaways
- ▪Midpoint execution manufactures fiction: Filling limit orders at mid-spread ignores natural price penalties, producing thousands of dollars in unearned paper returns.
- ▪Surface calibration is required: Raw ticks must be inverted through Black-Scholes-Merton or Bjerksund-Stensland to resolve Greeks when vendors omit volatility fields.
- ▪Physical settlement risk dwarfs cash index risk: In-the-money equity options trigger unexpected overnight share delivery, whereas European indices settle purely in cash without assignment risk.
- ▪Downside tails mandate non-normal metrics: Asymmetric return profiles render Sharpe ratios unreliable; engines must compute CVaR and Sortino ratios.
Structural Paradigm: Equity vs Derivatives Modeling
Scalar Equity Simulation
- • Evaluates a single continuous price vector: spot price trajectory over time (St).
- • Static ticker symbol universe with linear chronological transitions.
- • Negligible bid-ask drag; typical large-cap spreads remain within 1 to 2 basis points.
- • Exits cleanly realize cash proceeds without residual underlying obligations.
Options Surface Simulation
- • Evaluates high-dimensional matrices parameterized across strikes K, expiries τ, and rights ω.
- • Contracts expire, change moneyness dynamically, and experience non-linear Greek decay.
- • Wide execution spreads; option bid-ask widths routinely reach 5% to 30% of premium.
- • Introduces path-dependent corporate actions, early exercises, and overnight assignment shocks.
The Six-Stage Options Backtesting Pipeline
Derivatives simulation operates as a state machine tracking portfolio capital while synchronizing underlying spot assets with expiring contract chains.
| Pipeline Stage | Operational Preconditions | Execution Mechanism | System Output State |
|---|---|---|---|
| Chain Ingestion & Normalization | Raw EOD or intraday NBBO quote ticks/bars. | Standardize schema, purge crossed markets (Pb > Pa), invert implied volatility surface. | Parquet Chain Snapshot |
| Leg Discovery & Delta Targeting | Market snapshot, underlying spot St, strategy entry signal. | Moneyness bounds, root-finding for delta targets (Δ ≈ ∂V / ∂S), open interest verification. | Multi-Leg Contract Tuple |
| Execution Modeling & Margin Allocation | Target order intent, current bid-ask spread quotes. | Natural fills (sell bid, buy ask), spread-penalty friction, deduct Reg-T or Portfolio Margin. | Active Position & Buying Power Lock |
| Mark-to-Market & Position Lifecycle | Chronological quote snapshots, active positions. | Aggregate portfolio Greeks, mark net liquidation value, evaluate stop-loss and DTE limits. | Unrealized P&L & Exit Queue |
| Assignment, Exercise & Settlement | Active ITM contracts, ex-dividend calendars, expiration cutoff. | Ex-dividend assignment checks, OCC auto-exercise (≥ $0.01), cash settlement or stock delivery. | Realized Log & Assigned Inventory |
| Risk Diagnostics & Out-of-Sample Tests | Completed trade logs, margin utilization history, daily equity curve. | Asymmetric downside evaluation: Sortino, CVaR, Tail Ratio, Max Drawdown duration. | Risk Tearsheet & Sensitivity Map |
Surface Calibration Root-Finding
C_market − C_model(S_t, K, τ, r, q, σ) = 0- • Inverts Black-Scholes-Merton (European) or Bjerksund-Stensland (American).
- • Employs Brent's numerical root-finder to extract implied volatility σ.
- • Prunes crossed markets where Pbid > Pask.
Natural Execution with Spread Slippage
P_fill,buy = P_mid + α · ((P_ask − P_bid) / 2)- • α ∈ [0, 1] represents execution friction (α = 1.0 is full natural fill at the ask).
- • Natural execution sells at bid and buys at ask to eliminate unearned midpoint gains.
- • Deducts dynamic Regulation-T or Portfolio Margin allocations upon position entry.
Systematic Implementation via Optopsy
Developed by Michael Chu and distributed on GitLab, GitHub, and PyPI, Optopsy bifurcates derivative analysis into two operational modalities.
Vectorized Strategy Scanning
- • Evaluates historical chains globally using
op.iron_condor(). - • Identifies all trade setups matching target delta and DTE buckets across history simultaneously.
- • Generates aggregate win rates and payout distributions without simulating portfolio cash limits.
Chronological Portfolio Simulation
- • Runs step-by-step event simulation via
op.simulate()day by day. - • Enforces strict position concurrency caps (
max_positions) and capital bounds. - • Tracks multi-leg slippage, per-contract commissions, and dynamic DTE exit triggers.
import optopsy as op
chain_data = op.csv_data(
"spx_eod_chain_history.csv",
underlying_symbol=0, option_type=1, expiration=2,
quote_date=3, strike=4, bid=5, ask=6, underlying_price=7
)
simulation_result = op.simulate(
data=chain_data, strategy=op.iron_condor,
capital=100000.0, quantity=2, max_positions=3,
selector="nearest", max_entry_dte=45, exit_dte=14,
profit_target=0.50, stop_loss=2.00,
slippage_model="spread_pct", slippage_value=0.10,
commission_per_contract=0.65
)
performance_summary = simulation_result.summaryCompiled Engine Scaling: Optopsy-MCP
Rust + Apache Arrow + RhaiWhen scaling to high-frequency or minute-level resolutions, Python workflows face tabular join bottlenecks. The compiled optopsy-mcp engine loads Parquet partitions directly into memory-mapped Arrow tables, evaluating strategy logic inside an embedded Rhai scripting virtual machine.
fn on_bar(ctx) {
if ctx.position_count >= 3 { return []; }
if ctx.indicators.rsi < 35.0 {
let p = ctx.short_put(0.30, 45);
if p != () { return [p]; }
}
[]
}
fn on_exit_check(ctx, pos) {
if pos.pnl_pct >= 0.50 { return close_position("profit_target_reached"); }
if pos.dte <= 7 { return close_position("gamma_risk_mitigation"); }
hold_position()
}Scale and Structural Data Bottlenecks
In equity simulations, the data surface is a 1D vector. In options, the state space expands into a high-dimensional continuous and discrete manifold:
V: (t, K, T, ω, venue) → (bid, ask, bid_size, ask_size, σ, Δ, Γ, Θ, ν)| Universe & Granularity | Daily Records / Asset | Annual Size (1 Symbol) | US Market Annual Total |
|---|---|---|---|
| Equities: Daily EOD | 1 | ~10 KB | ~50 MB |
| Equities: 1-Minute Bars | 390 | ~4 MB | ~25 GB |
| SPX Options: Daily EOD Chain | 4,000–8,000 | ~50 MB | ~50 MB |
| SPX Options: 1-Minute Bars | 1.5M–3.0M | 15–30 GB | 15–30 GB |
| US Listed Options: 1-Min NBBO | ~400,000,000 | N/A | 2.5–4.0 TB (compressed) |
| US Listed Options: OPRA Tick | Tens of Billions | N/A | Multiple TBs / Day |
Partitioned Columnar Formats
Storage in Apache Parquet or Arrow, partitioned by symbol, year, and month. Allows projection pushdown to load only referenced strike/bid/ask columns.
Sparse Surface Grids
Interpolating across a pre-indexed grid of delta and maturity coordinates rather than performing full table scans over raw quotes.
Local Caches with Gap Detection
Maintaining local caches (~/.optopsy/cache/) that inspect Parquet bounds and download only missing historical trade dates.
Market Microstructure and Liquidity Traps
Options backtests routinely produce false alpha due to simplified execution pricing and reliance on stale trade logs.
The Midpoint Execution Trap
Phantom Alpha = 200 trades × 4 legs × $10.00 = $8,000.00- • An OTM put is quoted at $0.80 Bid / $1.00 Ask (Midpoint = $0.90).
- • Filling at midpoint captures an unearned $0.10/share ($10.00/contract) advantage on entry and another $10.00 on exit.
- • For an iron condor executing 200 trades annually across 4 legs, this manufactures $8,000.00 in fictitious profit.
- • On a $50,000 account, this distortion produces an artificial 16.0% annualized boost.
The Last Traded Price Fallacy
OTM strikes trade infrequently. If a stock plunges, quotes may widen to $4.20 / $4.60 while the recorded last_price remains at $0.35 from three days prior. Backtests relying on trade prints understate drawdowns and fail to trigger stop-losses.
Complex Orders vs Legging Risk
Multi-leg strategies trade in live markets via exchange Complex Order Books (COB). Simulating legs separately ignores wider composite spreads and obscures Legging Risk.
Assignment, Pin Risk, and Settlement Complexities
Options settlement introduces sharp structural divides between index products and physically delivered single equities.
| Settlement Dimension | Cash-Settled Indices (e.g., SPX, NDX) | Physically Settled Equities/ETFs (e.g., SPY, AAPL) |
|---|---|---|
| Settlement Rules | ITM value settled entirely in cash against the official SET quote. | Contracts ITM by ≥ $0.01 convert into 100 physical equity shares per contract. |
| Early Exercise Risk | Non-existent; European-style rules restrict exercise until expiration. | Persistent; highly acute prior to ex-dividend dates or when interest carry exceeds put extrinsic value. |
| After-Hours Market Exposure | Negligible; settlement determined at the calculation print. | High; underlying shares trade until 8:00 PM EST while OCC contrary exercise notices clear until 5:30 PM EST. |
| Pin Risk Mechanics | Limited to cash payout variance; zero overnight equity exposure. | Severe; unpredictable post-close assignment creates unhedged weekend stock positions. |
| Margin Shock Profile | Predictable; margin releases immediately upon cash settlement. | Catastrophic; physical share conversion can expand exposure tenfold, causing margin calls. |
Ex-Dividend Call Assignment Condition
D > C_extrinsic = C(S, K, τ) − (S − K)Rational counterparties exercise calls early when dividend D exceeds remaining extrinsic value, leaving short call sellers short stock and liable for the dividend.
Cost-of-Carry Put Exercise Condition
r · K · τ > P_extrinsicWhen interest rates r rise, the interest earned on cash strike proceeds K can exceed remaining put extrinsic value, triggering early assignment into long stock.
Pin Risk and the After-Hours Gap
If SPY closes at $500.02 at 4:00 PM EST, short 500 calls appear in-the-money by $0.02. However, long holders have until 5:30 PM EST to submit Contrary Exercise Advice. If negative earnings break at 4:30 PM dropping SPY to $496.00, long holders abandon their calls, leaving the options seller with unexpected short or unassigned equity exposure across the weekend.
Validation Protocols and Bias Elimination
Options strategies have numerous interacting parameters (DTE, target delta, wing width, profit target, stop-loss multiplier), making them acutely susceptible to curve-fitting.
Lookahead Contamination
- • Using EOD closing IV rank or ATR to enter morning trades at 10:00 AM.
- • Assuming an intraday profit target hit at the session low without checking if the high breached a stop-loss earlier.
- • Resolution: Enforce strict point-in-time state processing using lagged bars (t − Δt).
Survivorship Selection
- • Testing short-put strategies on modern S&P 500 index members across a 15-year lookback.
- • Excludes companies that suffered bankruptcy or distress-driven index removal.
- • Resolution: Ingest dynamic point-in-time index constituent listings that preserve delisted firms.
Out-of-Sample Verification Standards
Optimizes parameters on rolling in-sample windows (e.g., 24 months) and evaluates strictly out-of-sample (e.g., 6 months).
Purges training samples with overlapping contract lifecycles and embargos observations immediately following test shocks.
Ensures performance sits on a broad plateau; sharp returns at 18-delta that collapse at 17-delta or 19-delta indicate statistical noise.