Caskey Engineering

Lab

The math in use

Every equation below is implemented and running somewhere in this platform. Each entry links to the live page or names the module where it works.

Information Theory

Shannon entropy (normalized)

H=−∑ipilog⁡2pi,H^=Hlog⁡2NH = -\sum_{i} p_{i} \log_{2} p_{i}, \qquad \hat{H} = \frac{H}{\log_{2} N}

Measures how evenly a total is spread across its buckets, from all-in-one to perfectly even.

  • Backlog health tooling caskeycoding-specs _shared/tooling/backlog.py (health: epic distribution)

Exploring

Entropy-weighted digest diversity

H^=−∑spslog⁡ps,scorei∝ui⋅H^S∪{i}\hat{H} = -\sum_{s} p_{s} \log p_{s}, \qquad \text{score}_{i} \propto u_{i} \cdot \hat{H}_{S \cup \{i\}}

Ranks candidate picks higher when adding them would spread the digest across more distinct sections.

Opportunity: Rank Second Mind digest section diversity the way backlog health scores epic concentration.

Probability & Statistics

Isotonic regression (pool-adjacent-violators)

f^=arg⁡min⁡g nondecreasing∑iwi(yi−g(xi))2\hat{f} = \arg\min_{g \text{ nondecreasing}} \sum_{i} w_{i} \left( y_{i} - g(x_{i}) \right)^{2}

Fits the best stepwise curve that is only allowed to go up, never down.

  • Calibration engine ai-blog-api src/finance/engine/calibration.py

Brier score + Murphy decomposition

BS=1N∑i(pi−oi)2=REL−RES+UNC,UNC=oˉ(1−oˉ)BS = \frac{1}{N} \sum_{i} \left( p_{i} - o_{i} \right)^{2} = REL - RES + UNC, \qquad UNC = \bar{o} \left( 1 - \bar{o} \right)

Scores how far probability forecasts land from what actually happened, split into calibration, skill, and irreducible noise.

  • Forecast honesty scoring ai-blog-api src/finance/engine/calibration.py

Deflated Sharpe Ratio

DSR=Φ(SR−E[max⁡kSRk]σSR)DSR = \Phi \left( \frac{SR - E \left[ \max_{k} SR_{k} \right]}{\sigma_{SR}} \right)

Judges a strategy’s risk-adjusted return against the best result you would expect from luck alone after many trials.

  • Calibration engine + backtest stats ai-blog-api src/finance/engine/calibration.py + src/finance/backtest/stats.py

Probabilistic Sharpe Ratio + Minimum Track Record Length

PSR=Φ((SR−SR∗)n−11−γ3SR+γ4−14SR2),MinTRL=1+(ZαSR−SR∗)2(1−γ3SR+γ4−14SR2)PSR = \Phi \left( \frac{(SR - SR^{*}) \sqrt{n - 1}}{\sqrt{1 - \gamma_{3} SR + \frac{\gamma_{4} - 1}{4} SR^{2}}} \right), \qquad MinTRL = 1 + \left( \frac{Z_{\alpha}}{SR - SR^{*}} \right)^{2} \left( 1 - \gamma_{3} SR + \frac{\gamma_{4} - 1}{4} SR^{2} \right)

Says how confident you can be that a true edge beats a benchmark, and how long a track record you would need to prove it.

  • Backtest statistics ai-blog-api src/finance/backtest/stats.py

Probability of Backtest Overfitting (CSCV)

PBO=1S∑s1{ωs<0},ωs=log⁡(rs1−rs)PBO = \frac{1}{S} \sum_{s} \mathbb{1} \left\{ \omega_{s} < 0 \right\}, \qquad \omega_{s} = \log \left( \frac{r_{s}}{1 - r_{s}} \right)

Splits historical data many ways and counts how often the in-sample winner disappoints out of sample.

  • Backtest overfitting check ai-blog-api src/finance/backtest/pbo.py

Harvey-Liu-Zhu multiple-testing hurdle

E[max⁡t]=(1−γ)Φ−1(1−1M)+γ Φ−1(1−1Me)E \left[ \max t \right] = \left( 1 - \gamma \right) \Phi^{-1} \left( 1 - \frac{1}{M} \right) + \gamma \, \Phi^{-1} \left( 1 - \frac{1}{M e} \right)

Raises the bar a test statistic must clear the more strategies you have tried before it.

  • Calibration engine ai-blog-api src/finance/engine/calibration.py

Spearman rank information coefficient

IC=corr(rank(x),rank(y))IC = \text{corr} \left( \text{rank}(x), \text{rank}(y) \right)

Measures how well a signal’s ordering of stocks matches the ordering of what they actually returned.

  • Factor Monitor
  • Signal validation ai-blog-api src/finance/analysis/signal_validation.py

Stationary block bootstrap (Politis-Romano)

P(L=k)=(1−p)k−1pP \left( L = k \right) = \left( 1 - p \right)^{k - 1} p

Resamples a time series in random-length blocks so the shuffled copies keep the original’s rhythm.

  • Factor weighting ai-blog-api src/finance/analysis/scoring/weights_v2.py

Bootstrap percentile confidence bands

CIα=[θ^(α/2)∗,  θ^(1−α/2)∗]CI_{\alpha} = \left[ \hat{\theta}^{*}_{(\alpha / 2)}, \; \hat{\theta}^{*}_{(1 - \alpha / 2)} \right]

Brackets an estimate between the low and high percentiles of many resampled copies of the data.

Winsorization, z-score, MAD, rank-normalization

x′=min⁡(max⁡(x,qα),q1−α),z=x−μσ,zMAD=x−medMAD,x↦Φ−1(rank(x)n+1)x' = \min \left( \max \left( x, q_{\alpha} \right), q_{1 - \alpha} \right), \qquad z = \frac{x - \mu}{\sigma}, \qquad z_{MAD} = \frac{x - \text{med}}{MAD}, \qquad x \mapsto \Phi^{-1} \left( \frac{\text{rank}(x)}{n + 1} \right)

Four ways to put wildly different measurements on one shared scale without letting outliers dominate.

  • Seismograph
  • Signal normalization ai-blog-api src/finance/analysis/signals_normalize.py

Cross-sectional dispersion (population std dev)

σxs=1N∑i(ri−rˉ)2\sigma_{xs} = \sqrt{ \frac{1}{N} \sum_{i} \left( r_{i} - \bar{r} \right)^{2} }

Measures how far apart a group of returns are from each other at a single point in time.

Exploring

Embedding cosine similarity for semantic recall

sim(q,d)=q⋅d∥q∥ ∥d∥\text{sim}(q, d) = \frac{q \cdot d}{\lVert q \rVert \, \lVert d \rVert}

Finds notes that mean the same thing as the query even when they share no words with it.

Opportunity: Second Mind recall is lexical BM25 only; semantic nearest-neighbor over embeddings is the known gap.

Stochastic Processes

Black-Scholes price + Greeks

C=S0N(d1)−Ke−rTN(d2),d1=ln⁡(S0/K)+(r+σ2/2)TσT,d2=d1−σTC = S_{0} N(d_{1}) - K e^{-rT} N(d_{2}), \qquad d_{1} = \frac{\ln(S_{0} / K) + \left( r + \sigma^{2} / 2 \right) T}{\sigma \sqrt{T}}, \qquad d_{2} = d_{1} - \sigma \sqrt{T}

Prices a stock option from the stock price, strike, time, rate, and how jumpy the stock is.

Implied volatility (bracketed root-find)

σ^=arg⁡min⁡σ∈[σlo,σhi]∣CBS(σ)−Cmkt∣\hat{\sigma} = \arg\min_{\sigma \in [\sigma_{lo}, \sigma_{hi}]} \left\lvert C_{BS}(\sigma) - C_{mkt} \right\rvert

Works backwards from an option’s market price to the level of jumpiness the market is pricing in.

  • Implied-volatility surface ai-blog-web src/quant-viz/compute/impliedVol.ts

Geometric Brownian motion Monte Carlo

St+1=Stexp⁡((μ−σ22)Δt+σΔt Z),Z∼N(0,1)S_{t + 1} = S_{t} \exp \left( \left( \mu - \frac{\sigma^{2}}{2} \right) \Delta t + \sigma \sqrt{\Delta t} \, Z \right), \qquad Z \sim N(0, 1)

Simulates thousands of possible future price paths by stepping a random walk forward in time.

Realized volatility cone

σh=1n∑i=1nri2horizon h,Pq(h)=quantileq(σh)\sigma_{h} = \sqrt{ \frac{1}{n} \sum_{i = 1}^{n} r_{i}^{2} }_{\text{horizon } h}, \qquad P_{q}(h) = \text{quantile}_{q} \left( \sigma_{h} \right)

Shows how volatile the market has historically been at each lookback horizon, as a shaded cone of typical ranges.

Variance risk premium

VRP=σIV2−σRV2VRP = \sigma_{IV}^{2} - \sigma_{RV}^{2}

Compares the volatility investors are paying for with the volatility that actually showed up.

Exploring

Newton-Raphson implied vol with analytic vega

σk+1=σk−C(σk)−Cmktν(σk)\sigma_{k + 1} = \sigma_{k} - \frac{C(\sigma_{k}) - C_{mkt}}{\nu(\sigma_{k})}

Solves for implied volatility by repeatedly stepping along the slope of the price curve instead of bisecting a bracket.

Opportunity: Faster implied-vol solve than the current bracketed root-find if the surface ever uses a real option chain.

Time Series & Regimes

EWMA + standardized regime shift

μ^t=λμ^t−1+(1−λ)xt,zt=xt−μ^tσ^t\hat{\mu}_{t} = \lambda \hat{\mu}_{t - 1} + \left( 1 - \lambda \right) x_{t}, \qquad z_{t} = \frac{x_{t} - \hat{\mu}_{t}}{\hat{\sigma}_{t}}

Tracks a smoothed recent average and flags how unusual today’s value is against its own history.

  • Regime shift detection ai-blog-api src/finance/regime/shift.py

Absorption ratio (eigenvalue variance share)

AR=∑i=1nλi∑j=1NλjAR = \frac{\sum_{i = 1}^{n} \lambda_{i}}{\sum_{j = 1}^{N} \lambda_{j}}

Measures how much of a market’s movement is explained by just its top few common drivers.

  • Regime absorption ai-blog-api src/finance/regime/absorption.py

Nelson-Siegel yield curve

y(τ)=β0+β11−e−τ/λτ/λ+β2(1−e−τ/λτ/λ−e−τ/λ)y(\tau) = \beta_{0} + \beta_{1} \frac{1 - e^{-\tau / \lambda}}{\tau / \lambda} + \beta_{2} \left( \frac{1 - e^{-\tau / \lambda}}{\tau / \lambda} - e^{-\tau / \lambda} \right)

Draws a smooth curve through interest rates of every maturity using just a level, a slope, and a hump.

CPI year-over-year from index levels

YoYt=CPItCPIt−12−1YoY_{t} = \frac{CPI_{t}}{CPI_{t - 12}} - 1

Turns a monthly price index into the familiar annual inflation rate by comparing against the same month a year ago.

  • Macro Phase
  • Backend twin ai-blog-web src/quant-viz/compute/macroPhase.ts + ai-blog-api macro exporter (lockstep pair)

Percent above trailing mean (breadth)

Bt=1N∑i1{pi,t>pˉi}B_{t} = \frac{1}{N} \sum_{i} \mathbb{1} \left\{ p_{i, t} > \bar{p}_{i} \right\}

Counts what share of stocks are trading above their own recent average.

Simplified TRIMP training load

TRIMP=duration×intensityTRIMP = \text{duration} \times \text{intensity}

Scores a workout by multiplying how long it lasted by how hard it was.

  • Coach metrics engine ai-blog-api src/coach/metrics/engine.py

Acute:chronic workload ratio (ACWR)

ACWR=acute loadchronic loadACWR = \frac{\text{acute load}}{\text{chronic load}}

Compares this week’s training load against the recent average to flag doing too much too fast.

  • Coach metrics engine ai-blog-api src/coach/metrics/engine.py

Exponential half-life decay

w(d)=2−d/hw(d) = 2^{-d / h}

Weights recent items more than old ones, halving the weight every fixed number of days.

  • Digest ranking second-mind digest/build_digest.py

Exploring

Banister impulse-response (CTL/ATL/TSB)

CTLt=λfCTLt−1+(1−λf)Lt,ATLt=λgATLt−1+(1−λg)Lt,TSBt=CTLt−ATLtCTL_{t} = \lambda_{f} CTL_{t - 1} + \left( 1 - \lambda_{f} \right) L_{t}, \qquad ATL_{t} = \lambda_{g} ATL_{t - 1} + \left( 1 - \lambda_{g} \right) L_{t}, \qquad TSB_{t} = CTL_{t} - ATL_{t}

Models fitness as a slow-building response to training and fatigue as a fast-fading one, with form as the difference.

Opportunity: Coach has simplified TRIMP + ACWR only; fitness-fatigue modeling is the natural next tier.

Riegel race-time prediction

T2=T1(D2D1)αT_{2} = T_{1} \left( \frac{D_{2}}{D_{1}} \right)^{\alpha}

Predicts a race time at one distance from a known time at another distance.

Opportunity: Coach computes training load but predicts no race outcome.

Linear Algebra & Geometry

PCA via cyclic Jacobi eigendecomposition

A→RTAR,tan⁡2θ=2aijaii−ajjA \rightarrow R^{T} A R, \qquad \tan 2\theta = \frac{2 a_{ij}}{a_{ii} - a_{jj}}

Finds the main axes of variation in a dataset by repeatedly rotating the correlation matrix until its off-diagonal entries vanish.

Cosine similarity + mutual top-K graph + force layout

sij=xi⋅xj∥xi∥ ∥xj∥,Fi=∑j∈N(i)ks(dij−ℓ)u^ij−∑j≠ikrdij2u^ijs_{ij} = \frac{x_{i} \cdot x_{j}}{\lVert x_{i} \rVert \, \lVert x_{j} \rVert}, \qquad F_{i} = \sum_{j \in N(i)} k_{s} \left( d_{ij} - \ell \right) \hat{u}_{ij} - \sum_{j \neq i} \frac{k_{r}}{d_{ij}^{2}} \hat{u}_{ij}

Connects each point to its most similar neighbors and lets springs and repulsion spread them into a readable map.

Linear dynamical system + trace/determinant stability + RK4

x˙=Ax,stable  ⟺  tr(A)<0,  det⁡(A)>0,xt+1=xt+h6(k1+2k2+2k3+k4)\dot{x} = A x, \qquad \text{stable} \iff \text{tr}(A) < 0, \; \det(A) > 0, \qquad x_{t + 1} = x_{t} + \frac{h}{6} \left( k_{1} + 2 k_{2} + 2 k_{3} + k_{4} \right)

Classifies whether a simplified economic system settles down or spirals, and traces its path step by step.

OLS residualization / factor orthogonalization

β^=(XTX)−1XTy,r=y−Xβ^\hat{\beta} = \left( X^{T} X \right)^{-1} X^{T} y, \qquad r = y - X \hat{\beta}

Removes the part of a signal already explained by known factors, keeping only what is new.

  • Scoring pipeline ai-blog-api src/finance/analysis/scoring/pipeline_v2.py

Euclidean projection onto the probability simplex

w^=arg⁡min⁡w∈Δ∥w−v∥22,Δ={w:wi≥0,  ∑iwi=1}\hat{w} = \arg\min_{w \in \Delta} \lVert w - v \rVert_{2}^{2}, \qquad \Delta = \left\{ w : w_{i} \geq 0, \; \sum_{i} w_{i} = 1 \right\}

Adjusts a set of weights as little as possible so they are all non-negative and sum to one.

  • Portfolio benchmarks ai-blog-api src/finance/backtest/benchmarks.py

One-common-factor PSD correlation model

ρij=βiβj,∣βi∣≤1  ⟹  Σ⪰0\rho_{ij} = \beta_{i} \beta_{j}, \qquad \lvert \beta_{i} \rvert \leq 1 \implies \Sigma \succeq 0

Builds a valid correlation matrix for many assets from each one’s sensitivity to a single shared driver.

Optimization & Learning

Exponentiated-gradient multiplicative weights (Helmbold)

wt+1,i=wt,ieηz^t,i∑jwt,jeηz^t,jw_{t + 1, i} = \frac{w_{t, i} e^{\eta \hat{z}_{t, i}}}{\sum_{j} w_{t, j} e^{\eta \hat{z}_{t, j}}}

Rebalances a portfolio each period by multiplying each holding’s weight by a reward for its recent return.

  • Portfolio benchmarks ai-blog-api src/finance/backtest/benchmarks.py

OLMAR + best constant-rebalanced portfolio

x^t+1=xˉt+ϵmax⁡(0,xˉt−xt),w∗=arg⁡max⁡w∈Δ∏t(w⋅rt)\hat{x}_{t + 1} = \bar{x}_{t} + \epsilon \max \left( 0, \bar{x}_{t} - x_{t} \right), \qquad w^{*} = \arg\max_{w \in \Delta} \prod_{t} \left( w \cdot r_{t} \right)

Buys what fell below its recent average expecting a snap back, judged against the best fixed mix in hindsight.

  • Portfolio benchmarks ai-blog-api src/finance/backtest/benchmarks.py

Kelly criterion

f∗=p−1−pbf^{*} = p - \frac{1 - p}{b}

Sizes a bet to grow wealth fastest over the long run given your edge and the payoff odds.

  • Position sizing ai-blog-api src/finance/engine/sizing.py

Volatility targeting + drawdown throttle

wt=σ∗σ^t,wt↦wt⋅g(DDt)w_{t} = \frac{\sigma^{*}}{\hat{\sigma}_{t}}, \qquad w_{t} \mapsto w_{t} \cdot g \left( DD_{t} \right)

Scales exposure down when risk runs hot and eases off further after losses pile up.

  • Position sizing ai-blog-api src/finance/engine/sizing.py

ICIR factor weighting with James-Stein shrinkage

wi∝ICiσICi,θ^=(1−(k−2)σ2∥xˉ∥2)+xˉw_{i} \propto \frac{IC_{i}}{\sigma_{IC_{i}}}, \qquad \hat{\theta} = \left( 1 - \frac{(k - 2) \sigma^{2}}{\lVert \bar{x} \rVert^{2}} \right)^{+} \bar{x}

Weights signals by how consistently they have predicted returns, pulled toward zero to avoid over-trusting small samples.

  • Factor weighting ai-blog-api src/finance/analysis/scoring/weights_v2.py

BM25 relevance ranking

score(D,q)=∑iIDF(qi) f(qi,D)(k1+1)f(qi,D)+k1(1−b+b∣D∣avgdl)\text{score}(D, q) = \sum_{i} IDF(q_{i}) \, \frac{f(q_{i}, D) \left( k_{1} + 1 \right)}{f(q_{i}, D) + k_{1} \left( 1 - b + b \frac{\lvert D \rvert}{avgdl} \right)}

Ranks documents by how well their words match a query, rewarding rare terms and not over-crediting long documents.

  • Second Mind search second-mind mcp/src/db.js (SQLite FTS5 bm25())

Budgeted greedy selection under byte cost

max⁡S∑i∈Suis.t.∑i∈Sci≤B,pick by ui/ci\max_{S} \sum_{i \in S} u_{i} \quad \text{s.t.} \sum_{i \in S} c_{i} \leq B, \qquad \text{pick by } u_{i} / c_{i}

Fills a fixed-size digest by repeatedly taking the item with the most value per byte it costs.

  • Digest packing second-mind digest/build_digest.py

Grade banding with hysteresis

gt={upgradest>θuholdθd≤st≤θudowngradest<θdg_{t} = \begin{cases} \text{upgrade} & s_{t} > \theta_{u} \\ \text{hold} & \theta_{d} \leq s_{t} \leq \theta_{u} \\ \text{downgrade} & s_{t} < \theta_{d} \end{cases}

Changes a grade only when the score crosses a clearly higher or lower band, so borderline scores stop flapping.

  • Composite grading ai-blog-api src/finance/analysis/scoring/composite.py

Numerical Methods

Acklam inverse normal CDF approximation

Φ−1(p)≈∑kaktk1+∑kbktk,t=−2ln⁡p(central region)\Phi^{-1}(p) \approx \frac{\sum_{k} a_{k} t^{k}}{1 + \sum_{k} b_{k} t^{k}}, \qquad t = \sqrt{-2 \ln p} \quad \text{(central region)}

Converts a probability back into the number of standard deviations from the mean, with a fast rational approximation.

  • Calibration engine ai-blog-api src/finance/engine/calibration.py

Piecewise linear interpolation

f(x)=yi+x−xixi+1−xi(yi+1−yi),x∈[xi,xi+1]f(x) = y_{i} + \frac{x - x_{i}}{x_{i + 1} - x_{i}} \left( y_{i + 1} - y_{i} \right), \qquad x \in [x_{i}, x_{i + 1}]

Estimates values between known points by connecting them with straight lines.

  • Today
  • Interpolation utilities ai-blog-api src/finance/utils/interpolation.py

Equal-width histogram binning

b(x)=min⁡(⌊x−xmin⁡w⌋,  K−1),w=xmax⁡−xmin⁡Kb(x) = \min \left( \left\lfloor \frac{x - x_{\min}}{w} \right\rfloor, \; K - 1 \right), \qquad w = \frac{x_{\max} - x_{\min}}{K}

Sorts values into equal-width buckets to draw the shape of a distribution.

Seeded PRNG discipline (mulberry32, hash jitter)

seed↦{un},un∈[0,1)seed \mapsto \left\{ u_{n} \right\}, \qquad u_{n} \in [0, 1)

Makes every random-looking picture come out identical every time from a fixed starting seed.

  • All stochastic visualizations ai-blog-web src/quant-viz/compute (all stochastic paths)