The "Dust" Problem
Traditional Mean-Variance Optimization assumes assets are infinitely divisible. This creates "dust"—negligible positions (e.g., 0.0001%) that are costly to trade, operational nightmares, impossible to hedge, and illiquid odd-lots.
The MIP Solution
We introduce a binary vector z where z_i ∈ {0, 1}. If z_i is 0, weight w_i MUST be 0.
Mixed-Integer Programming (MIP)
Mathematical Architectures
Defining the feasible region is an art form. We move beyond simple linear bounds to capture the discrete nature of trading mechanics and solver performance.
Structural Constraints
Logical Constraints
Encodes "If-Then" rules. Example: "If we hold Shell, we must not hold BP."
Cardinality (K)
Limits the total number of assets in the portfolio to exactly K.
Minimum Buy-In
Disallows small trades. Position must be 0 or > $100k.
Round Lots
Forces trades to be multiples of a lot size (e.g., 100 shares).
Advanced Techniques
Perspective Cut
Advanced conic reformulation. Replaces quadratic terms to tighten the "relaxation gap."
Indicator Constraints
Modern solver feature. Avoids "Big-M" numerical issues by handling logic natively.
SOS Type 2
Special Ordered Sets. Essential for modeling piecewise linear costs.
Turnover Control
Linearizing absolute value differences for rebalancing limits.
Research Note: The "Big-M" Pitfall
When linking binary (z) and continuous (w) variables via w ≤ M · z, choosing a generic "large M" (e.g., 10,000) causes numerical instability in solvers. A "Tight M" (equal to the asset's upper bound) is critical for convergence.
The Quant Workflow
1. Data Ingestion & Signal Generation
Constructing the inputs for the optimizer using Python/Pandas.
- Expected Returns (μ): Alpha model output. Vector of size N.
- Covariance Matrix (Σ): Risk model (e.g., Barra). Matrix of size N×N.
2. Problem Formulation (CVXPY)
Translating business logic into standard form via MIP Modeling.
3. The Solver Engine
Branch-and-Bound search space exploration (e.g., Gurobi, Mosek).
4. Order Slicing & Execution
Transforming optimal weights into market orders via FIX Protocol.
- Round to nearest Lot (100)
- Split large parents (VWAP)
- Route to Dark Pools
- TCA Analysis
Strategic Applications
Sparse Index Tracking
The goal is to replicate a benchmark (e.g., S&P 500) using only a subset of assets (e.g., K=50). This minimizes transaction costs and simplifies management.
Lasso (L1)
Shrinks weights towards zero. Bias creates systematic underperformance.
MIP (L0)
Selects the best subset without shrinking weights. Provides an unbiased estimator.
Tax-Loss Harvesting
Systematically realizing losses to offset capital gains, while maintaining risk exposure. The complexity lies in the Wash Sale Rule: you cannot buy a "substantially identical" security 30 days before or after a sale.
Wash Sale Constraint (MIP)
The Modern Quant Stack
Modeling
- CVXPY: Python DSL for convex optimization. The industry standard.
- JuMP: Julia-based modeling. Extremely fast for large-scale problems.
Engines
- Gurobi: Best-in-class performance for MIPs. Expensive licensing.
- HiGHS: High-performance open-source linear solver (C++).
Data & Infra
- kdb+ / q: Time-series database for high-frequency tick data.
- Kubernetes/Airflow: Orchestrating distributed solver jobs and daily rebalancing DAGs.
Future Frontiers
Quantum Annealing
Classical solvers struggle with non-convex landscapes, often getting stuck in local minima. Quantum Annealers exploit quantum tunneling to traverse energy barriers, finding global optima for combinatorial problems.
QUBO Formulation
Financial MIPs must be reformulated into Quadratic Unconstrained Binary Optimization problems.
- Logical Variables Qubits
- Correlations Couplers (J)
- Returns/Risk Bias (h)
- Constraints Penalty Terms
Neural Branching
The bottleneck of any MIP solver is the Branch-and-Bound tree. Choosing which variable to branch on determines if the solver finishes in seconds or centuries.
We train Graph Neural Networks (GNNs) via Imitation Learning to mimic expert (but slow) branching rules like Strong Branching, but execute them in milliseconds on a GPU.