Deflated Sharpe Ratio (DSR)
A correction of the observed Sharpe ratio for multiple testing, non-normal returns, and observation window length. Threshold at Gate 1a is DSR ≥ 0.95 (return-generating strategies; per Doctrine §9.1 bifurcation).
The Deflated Sharpe Ratio, published by Bailey and López de Prado in the Journal of Portfolio Management in 2014, corrects an observed Sharpe ratio for three biases that inflate it in ordinary use: multiple testing, non-normality of returns, and the length of the observation window. A DSR value near 1.0 means the observed Sharpe would survive correction almost certainly; a DSR below 0.5 means the observed Sharpe is likely a statistical artifact of the search process rather than a true edge. In Nyalai's five-gate framework, the DSR is the test used at Gate 1a (statistical fragility for return-generating strategies) per Doctrine §9.1. Probabilistic prediction systems are evaluated at Gate 1b using the Brier Skill Score instead.
Common misuse
Quoting a raw Sharpe as evidence of edge without disclosing the number of configurations tested during development. Without that number, the DSR cannot be computed honestly, and the raw Sharpe is uninterpretable.
Cross-references:Gate 1 · guided-search bias · statistical inference horizon
Brier Skill Score (BSS)
A comparison of a probabilistic forecast to a naïve baseline. Positive BSS means calibration skill; negative means worse than the base rate. Threshold BSS ≥ 0.05 applies at Gate 1b (probabilistic prediction systems, statistical fragility) and at Gate 4 (probabilistic calibration).
The Brier Skill Score compares the calibration of a probabilistic forecast to a naïve baseline (typically the base rate). It ranges from negative infinity to 1. A BSS of 0 means the forecast is no better than the naïve baseline. A BSS above 0 means the forecast adds calibration skill; a BSS below 0 means the forecast is worse than saying always predict the base rate. In Nyalai's framework, the BSS carries at two gates. At Gate 1b (statistical fragility for probabilistic prediction systems, per Doctrine §9.1) the BSS tests aggregate skill in excess of chance after exploration correction. At Gate 4 (probabilistic calibration) the BSS sits alongside the Expected Calibration Error to test whether emitted probabilities match realized frequencies. The two gates share the same 0.05 floor by numeric coincidence and by legibility choice, not by double jeopardy: they measure different failure modes. The threshold is BSS ≥ 0.05. Origin: Glenn Brier, Verification of Forecasts Expressed in Terms of Probability, Monthly Weather Review, 1950.
Common misuse
Reporting a positive BSS on the training data and treating it as evidence of forecasting skill in production. BSS on the training data is inflated by the fit; only out-of-sample BSS carries information.
Cross-references:Gate 1b · Gate 4 · Expected Calibration Error · probabilistic calibration
Expected Calibration Error (ECE)
The average absolute gap between a forecast's stated confidence and its realized frequency across confidence bins. Threshold at Gate 4 is ECE ≤ 0.05.
The Expected Calibration Error measures the average absolute gap between a forecast's stated confidence and its realized frequency, aggregated across confidence bins. If a forecast repeatedly says 70% confidence but is right only 55% of the time, the ECE captures that miscalibration. In Nyalai's framework, the ECE is the second test at Gate 4 (probabilistic calibration). A forecast can be directionally correct (right more often than wrong) but severely miscalibrated (its confidence numbers do not match its accuracy). Miscalibrated confidence breaks position sizing, since Kelly-style or risk-parity allocation depends on the confidence estimate being trustworthy.
Common misuse
Assuming a well-calibrated forecast on the training window remains calibrated in production. Distributional drift is the most common cause of ECE degradation.
Cross-references:Gate 4 · Brier Skill Score · Kelly criterion
Hidden Markov Model (HMM)
A statistical framework for describing systems where observable data is generated by a latent state that transitions probabilistically. Used for regime detection at Gate 2.
A Hidden Markov Model is a statistical framework for describing systems where observable data is generated by a latent (unobservable) state that transitions probabilistically. In quantitative finance, HMMs are typically used to detect regime changes in market data: the observable is the return series, the latent state is the regime (for example bull, sideways, bear, or more granular states inferred from data). HMMs require choosing the number of latent states and estimating the transition matrix from data using algorithms like Baum-Welch. Once fit, an HMM can label each historical period with its most-probable regime, which allows testing strategy performance conditional on regime. In Nyalai's framework, HMM regime detection is one operationalization of Gate 2 (cross-regime robustness).
Common misuse
Fitting an HMM with too many states, then claiming that a strategy passes Gate 2 because it has positive Sharpe in every micro-regime. The number of states is itself a hyperparameter subject to guided-search discipline.
Cross-references:Gate 2 · cross-regime robustness · Baum-Welch · latent state
Bootstrap block size
The length of consecutive time-series observations resampled together as a unit in block bootstrap. Chosen to preserve autocorrelation structure at Gate 5.
In block bootstrap resampling (Politis and Romano, 1994), the block size is the length of consecutive time-series observations that are resampled together as a unit. Block bootstrap is used to preserve the autocorrelation structure of the original series when generating synthetic paths for confidence interval estimation. Choosing block size correctly is non-trivial. Too small and the bootstrap destroys the autocorrelation. Too large and the bootstrap loses variability. The econometric literature offers criteria based on estimated autocorrelation decay. In Nyalai's framework, block bootstrap is the test used at Gate 5 (bootstrap confidence). The chosen block size is disclosed in the reproducibility appendix of every verdict-form.
Common misuse
Using a fixed block size (say 20 or 60 days) without justification against the specific return series being tested.
Cross-references:Gate 5 · autocorrelation · reproducibility appendix
Guided-search bias
Inflation of a strategy's apparent significance caused by directed exploration during development. Corrected at Gate 3 by the Aronson-Masters permutation test.
Guided-search bias is the inflation of a strategy's apparent statistical significance caused by the developer having tested many variants and retained the one that performed best. It differs from ordinary multiple-testing bias in that the search is directed by the results of prior tests, not random. Machine learning workflows, Optuna hyperparameter sweeps, walk-forward configuration searches, and iterative feature engineering all introduce guided-search bias by default. The Aronson-Masters permutation test (2013) is designed to correct significance estimates for the disclosed number of configurations tested. In Nyalai's framework, Gate 3 (guided-search safety) uses this correction.
Common misuse
Disclosing the number of configurations tested in the final ensemble but omitting nested tuning layers (feature selection passes, hyperparameter grids per configuration). All layers of guided search must be counted.
Cross-references:Gate 3 · Aronson-Masters test · Deflated Sharpe Ratio · Optuna
Statistical inference horizon
The length of historical data needed for a claim about a return distribution to be reliably estimated. In financial time series, uncomfortably long relative to how long the claim needs to hold.
The statistical inference horizon is the length of historical data needed for a claim about a distribution or a return premium to be reliably estimated. In financial time series, this horizon is uncomfortably long compared to the horizon over which the underlying claims need to hold. Fama and French's 2018 paper Volatility Lessons (Financial Analysts Journal) documents that even ten-year rolling windows admit substantial probability of misleading realized premia. In other words, ten years of data may not be enough to distinguish a real edge from a lucky path. In Nyalai's framework, the statistical inference horizon is invoked at Section 2.7 of the Refusal Doctrine to justify why external adversarial validation is necessary rather than optional.
Common misuse
Claiming twenty years of data as sufficient to validate a strategy without examining what fraction of that twenty years is genuinely out-of-sample versus contaminated by prior in-sample optimization.
Cross-references:Fama-French · Section 2.7 · Fourth Quadrant · sequence commitment
Regime detection
Identification of distinct statistical states in a market time series, so strategy performance can be evaluated conditional on regime rather than only on aggregate.
Regime detection is the identification of distinct statistical states in a market time series, such that strategy performance can be evaluated conditional on regime rather than only on aggregate. The rationale is that many strategies profitable in aggregate are in fact concentrated in a single regime and lose money in others. Regime detection can be explicit (rolling percentile buckets, threshold-based classifiers on realized volatility or trend) or inferred from data via Hidden Markov Model or other latent-state approaches. Both are legitimate provided the choice is disclosed and the number of regimes is justified. In Nyalai's framework, Gate 2 (cross-regime robustness) requires Sharpe positive in every detected regime.
Common misuse
Using aggregate-window Sharpe to claim regime robustness without explicitly decomposing performance by regime.
Cross-references:Gate 2 · Hidden Markov Model · latent state · cross-regime robustness
Latent state
A variable that governs observable behavior but is not directly measured. The market regime is the canonical latent state in quantitative finance.
A latent state is a variable that governs the observable behavior of a system but is not directly observable. In quantitative finance, the market regime is the canonical latent state: the return series is observable, but the underlying regime (bull, bear, sideways, or finer granularity) is inferred, not directly measured. Hidden Markov Models are the standard framework for latent-state inference in time series. The number of latent states, the transition matrix between them, and the observable distribution conditional on each state are all estimated from data.
Common misuse
Treating an HMM output as if the latent states have real-world semantic meaning (interpreting state 3 as bullish) without empirical validation of that interpretation.
Cross-references:Hidden Markov Model · Gate 2 · Baum-Welch
Sovereignty rule
Nyalai's structural requirement of separation between strategy producer and strategy evaluator. Formalized in Section 5 and Section 10 of the Refusal Doctrine.
The sovereignty rule is Nyalai's requirement of structural separation between the party developing a strategy and the party evaluating it. Formalized in Section 5 and Section 10 of the Refusal Doctrine, the rule prohibits Nyalai from co-developing strategies with the entities it validates, from taking equity in submitters, from earning fees contingent on verdict outcomes, and from sharing operational upside with model owners. The rule is not a matter of professional preference. It is a structural constraint that makes Nyalai's verdicts independent by construction rather than by trust. In practice: personal friendships are disclosed as public record. The friendship is not a disqualification; the absence of shared economic upside is what preserves independence.
Common misuse
Assuming that we know each other equals we cannot audit each other. The doctrine distinguishes personal relationship (permitted, disclosed) from shared economic upside (prohibited).
Cross-references:Section 5.3 · Section 10 · sequence commitment
Verdict-form
Nyalai's canonical output document. Reports verdict (CALIBRATED or NOT-CALIBRATED), gate arithmetic, scope statement, triangulation panel, and reproducibility appendix. Published under CC0.
A verdict-form is Nyalai's canonical output document: a structured markdown or web page that reports the verdict (CALIBRATED or NOT-CALIBRATED), the gate-by-gate arithmetic, the verdict scope statement, the triangulation panel, and the reproducibility appendix. Every verdict-form is signed, timestamped, hash-anchored, and published under CC0. Reference R-01 (the illustrative CALIBRATED case) and Verdict-form 001 on Synapse v0.3 (NOT-CALIBRATED) are the first two live examples. The verdict-form is designed for institutional citation. Model owners, independent validators, internal audit, and regulators can all reference the same document from their respective institutional roles.
Common misuse
Treating a verdict-form as an investment recommendation. A CALIBRATED verdict is a statement about model calibration on a specific ledger; it is not a signal to allocate capital.
Cross-references:Verdict scope statement · Model Trust Level · sequence commitment
Model Trust Level (MTL)
Nyalai's five-tier taxonomy for how much trust to place in a probabilistic prediction system. MTL-1 Untested through MTL-5 Continuously Live-Validated. Strictly nested.
Model Trust Level is Nyalai's five-tier taxonomy for the amount of trust an allocator should place in a probabilistic prediction system. MTL-1 is Untested; MTL-2 is Nominally Calibrated (Gates 1 and 4 passed); MTL-3 is Robustly Calibrated (adds Gate 2); MTL-4 is Adversarially Validated (all five gates plus sovereignty rule enforced); MTL-5 is Continuously Live-Validated (adds rolling drift detection). The levels are strictly nested: MTL-N includes every requirement of every lower level. Each MTL comes with a recommended deployment context.
Common misuse
Attributing an MTL without stating the doctrine version and the evaluation date. MTLs are time-bounded; they expire under style drift or ledger extension.
Cross-references:Refusal Doctrine Section 10 · style drift · verdict scope statement
Fourth Quadrant (Taleb)
Taleb's region of decision-making where complex payoffs meet fat-tailed distributions. Where standard statistical inference breaks down and adversarial validation becomes necessary.
The Fourth Quadrant, articulated by Nassim Taleb in a 2008 Edge.org essay and its 2009 International Journal of Forecasting follow-up, is the region of decision-making where complex payoffs (nonlinear, unbounded loss possibilities) meet fat-tailed distributions (regime shifts, jumps). It is the region where standard statistical inference breaks down. Institutional model risk management, as practiced under SR 11-7 and equivalent guidance, addresses the first three quadrants of decision-making adequately. The Fourth Quadrant is where standard practice fails and external adversarial validation becomes necessary. In Nyalai's framing, the Fourth Quadrant is Section 1.4 of the Refusal Doctrine.
Common misuse
Treating the Fourth Quadrant as a warning to avoid rather than as a region requiring specialized validation. Nyalai's position is that the Fourth Quadrant is where the interesting problems are and must be entered with the right instruments.
Cross-references:Taleb · Section 1.4 · statistical inference horizon
Sequence commitment
Nyalai's order-of-operations rule around input integrity. Submissions frozen before downstream use is discussed. Consent collected after the verdict is rendered.
The sequence commitment is Nyalai's order-of-operations rule around input integrity: submissions are frozen (timestamped and hashed) before any conversation about downstream use of the verdict occurs. Consent for publication or downstream reference is collected after the verdict is rendered, on a document the submitter reviews. Refusal of downstream use does not alter the verdict itself. The purpose is to prevent the class of failure mode in which the submitter learns what the verdict will be before consenting to publish it, then withholds consent when the verdict is unfavorable. Formalized in Section 12.4 of the Refusal Doctrine.
Common misuse
Assuming I asked for a private evaluation, so the result stays private satisfies sequence commitment. It does not. The evaluation is frozen at submission; only publication path is subject to consent.
Cross-references:Refusal Doctrine Section 12.4 · verdict-form · sovereignty rule
Triangulation panel
Nyalai's three-lens contextualization of every verdict. Risk type times business type times regulatory frame. Adapted from Senthil Kumar's framework at BNY Mellon.
The triangulation panel is the three-lens framework Nyalai adopts from Senthil Kumar's institutional risk management practice (formerly CRO at BNY Mellon, now CRO at Huntington Bank). Every verdict-form contextualizes the verdict across three dimensions: risk type exposed to the allocator (credit, market, operational, liquidity, model, counterparty); business type against which the verdict is calibrated (institutional multi-manager pool, single-strategy fund, family office, etc.); and regulatory frame at the deployment location (SR 11-7, Solvency II, ACPR, ISDA SIMM, etc.). The purpose is to prevent misapplication of the verdict outside the cell of triangulation for which it was calibrated. Formalized in Section 9.11 of the Refusal Doctrine.
Common misuse
Reading the executive summary of a verdict-form and applying the verdict outside its triangulation cell without re-evaluation. The triangulation panel is a warning label, not a decorative section.
Cross-references:Senthil Kumar · Section 9.11 · verdict scope statement
