SOPHIE Daddy Quant Blog - Stock & Options Analysis

Core Theses

Predictive models relying exclusively on historical correlations degrade violently across regime changes.

Double Machine Learning neutralizes regularization bias, enabling unbiased causal parameter estimation in high-dimensional noise.

Continuous optimization (NOTEARS) and non-Gaussian frameworks (LiNGAM) can mathematically learn structural graphs directly from observational data.

✓

Replacing the Pearson matrix with the Interventional Covariance Matrix dramatically improves out-of-sample Sharpe ratios.

The Epistemological Crisis in Empirical Finance

•
Historically, quantitative finance relied on associational statistics (e.g., CAPM, Fama-French, standard deep learning).
•
Without causal frameworks, mining thousands of factors mathematically guarantees finding statistically significant but spurious correlations, artificially inflating the False Discovery Rate (FDR).
•
An investment strategy devoid of a formal causal theory is highly likely to be mathematically false out-of-sample.

Associational Inference (Seeing)

•Measured by Conditional Probability: $P(Y \mid X)$
•Relies on Undirected / Correlational graphs.
•Vulnerable to Confounding, Colliders, and Simpson's Paradox.
•Degrades violently during regime shifts.

Causal Inference (Doing)

•Measured by Do-Calculus: $P(Y \mid \operatorname{do}(X))$ via SCMs.
•Relies on Directed Acyclic Graphs (DAGs) to map temporal flow.
•Actively mitigates spurious "factor mirages".
•Isolates invariant features robust to market turbulence.

Featured Infographic

The structural revolution in quantitative finance: causal inference vs correlational models

Theoretical Foundations: Graphing Market Structure

A Directed Acyclic Graph (DAG) maps the topological information of a system. Understanding this geometry is required to avoid destructive biases:

Confounding Bias

An unobserved variable causes both the treatment and the outcome. Failing to condition on it creates a spurious association.

Collider Bias

The treatment and the outcome both cause a third variable. Conditioning on a collider induces artificial correlation between independent variables.

Pearl's Backdoor Adjustment (Do-Calculus)

P(Y \mid \operatorname{do}(T = t)) = \sum_{X} P(Y \mid T = t, X = x) P(X = x)

do(T=t)=Interventional operator setting treatment T to value t (severing incoming arrows)

X=Confounder set satisfying the Backdoor Criterion relative to (T, Y)

P(Y|do(T))=Causal interventional probability distribution isolated from confounding bias

High-Dimensional Parameter Estimation: Double Machine Learning

•
Standard ML architectures (Random Forests, DNNs) suffer from severe regularization bias, systematically shrinking causal parameter estimates toward zero and ruining inference.
•
Double Machine Learning (DML) neutralizes regularization bias, enabling the estimation of specific causal parameters amid high-dimensional nuisance covariates.

Double Machine Learning (DML) Orthogonalized Estimator

\hat{\theta} = \left( \frac{1}{n} \sum_{i=1}^n \tilde{D}_i^2 \right)^{-1} \left( \frac{1}{n} \sum_{i=1}^n \tilde{D}_i \tilde{Y}_i \right)

Y=Outcome asset return: Y = θ₀ D + g₀(X) + U with E[U|X,D] = 0

D=Treatment shock variable: D = m₀(X) + V with E[V|X] = 0

X=High-dimensional confounding covariates (macro, factor exposures)

Ỹ, D̃=Out-of-fold cross-fitted residuals: Ỹ = Y - ĝ(X), D̃ = D - m̂(X)

θ̂=Neyman-orthogonal causal effect achieving √n-consistency

Step 1: Cross-Fitting

Split data into K folds. Train arbitrary ML models to predict $Y$ from $X$ ( $\hat{g}(X)$ ) and $D$ from $X$ ( $\hat{m}(X)$ ) out-of-sample.

Step 2: Residualization

Calculate residuals $\tilde{Y} = Y - \hat{g}(X)$ and $\tilde{D} = D - \hat{m}(X)$ . This isolates the variation in $D$ independent of confounders $X$ .

Step 3: Neyman Estimation

Regress $\tilde{Y}$ on $\tilde{D}$ . The Neyman condition guarantees $\sqrt{n}$ -consistency even when ML nuisance functions converge at slower $n^{-1/4}$ rates.

Tackling Unobserved Confounding: Deep IV

•
When confounders (like hidden macro sentiment or liquidity shocks) are unobservable, researchers use Instrumental Variable (IV) regression.
•
Deep IV frameworks integrate deep neural networks into IV regression, modeling highly non-linear asset pricing structures while maintaining econometric unconfoundedness.

Deep Instrumental Variables (Deep IV) Formulation

\mathbb{E}[Y \mid Z, X] = \int g(d, X) \, dP(d \mid Z, X)

Z=Exogenous instrument (satisfying relevance, exclusion, and unconfoundedness)

D=Endogenous treatment variable subject to unobserved confounding

X=Observable background conditioning covariates

g(d, X)=Non-linear structural causal response function parameterized via neural networks

Causal Discovery: Inferring Structural Graphs

Causal Discovery mathematically learns the DAG directly from observational data, categorized into distinct algorithmic families:

Family	Mechanism	Quant Advantage
Constraint-Based (PC, FCI)	Conditional independence tests.	FCI mathematically handles unobserved confounders.
Continuous Optimization (NOTEARS)	Differentiable algebraic constraint.	Scales to large universes; integrates with NNs.
Non-Gaussian (LiNGAM)	Independent Component Analysis.	Orients causal arrows uniquely, resolving equivalence.

Continuous Acyclicity Constraint (NOTEARS)

\min_{W \in \mathbb{R}^{d \times d}} \frac{1}{2n} \|X - X W\|_F^2 + \lambda \|W\|_1 \quad \text{s.t.} \quad h(W) = \operatorname{tr}\left(e^{W \circ W}\right) - d = 0

W=Weighted adjacency matrix of directed acyclic graph (DAG)

W ∘ W=Hadamard (element-wise) matrix product

tr(e^A)=Trace of matrix exponential enforcing exact graph acyclicity

d=Number of assets / variables in the target universe

Linear Non-Gaussian Acyclic Model (LiNGAM)

X = B X + e = (I - B)^{-1} e, \quad e_i \sim \text{Non-Gaussian}

B=Strictly lower-triangular DAG adjacency matrix under topological ordering

e=Mutually independent non-Gaussian error terms (exploited via ICA)

The Role of LLMs

LLMs act as "causal parrots" hallucinating relationships via linguistic co-occurrence. They must never independently arbitrate causality. Instead, they supply directional priors (soft constraints) to algorithms like NOTEARS, demonstrably boosting F1 accuracy on synthetic financial graphs by over 300%.

Causal Portfolio Analytics: Rethinking the Covariance Matrix

•
Modern Portfolio Theory relies on the Pearson matrix, capturing spurious correlations that notoriously converge toward 1.0 during market crashes.
•
The Interventional Covariance Matrix ( $\Sigma_{do}$ ) isolates structural dependencies by controlling for shared causal ancestors.
✓
Portfolios optimized on $\Sigma_{do}$ exhibit vastly superior out-of-sample Sharpe ratios and faster drawdown recoveries because they rely on invariant structures.

The Interventional Covariance Matrix (Σ_do)

\Sigma_{do} = (I - B)^{-1} \left( \Gamma \Sigma_F \Gamma^T + \Sigma_\epsilon \right) ((I - B)^{-1})^T

B=Topological DAG causal adjacency matrix among asset returns

Γ=Factor loading matrix on common causal macroeconomic drivers F

Σ_F=Covariance matrix of exogenous macroeconomic drivers

Σ_ε=Diagonal idiosyncratic noise variance matrix

Σ_do=Interventional covariance isolating true invariant structural dependencies

Causal Mean-Variance Portfolio Optimization

Replacing the fragile Pearson sample covariance matrix with $\Sigma_{do}$ generates portfolios shielded from spurious correlation collapses during market panics:

\min_{\mathbf{w}} \mathbf{w}^T \Sigma_{do} \mathbf{w} \quad \text{s.t.} \quad \mathbf{w}^T \mathbf{\mu} \ge \mu_{\text{target}}, \quad \mathbf{w}^T \mathbf{1} = 1

The Structural Revolution in Causal Quantitative Finance