Methodology · one page
Five adversarial gates. One binary verdict.
Nyalai returns a CALIBRATED or NOT-CALIBRATED verdict on every probabilistic system submitted for validation. Each gate uses the statistical framework appropriate to its specific question. A strategy that clears all five is Adversarially Validated. A strategy that fails one is refused.
The problem
Institutional MRM addresses the first three quadrants. Nyalai closes the fourth.
In the taxonomy Nassim Taleb published in 2008, decisions divide along two axes: whether the payoff structure is simple or complex, and whether the underlying randomness lives in thin-tailed or fat-tailed distributions. Quantitative strategies operate in the complex-and-fat-tailed region: nonlinear payoffs, unbounded loss possibilities, regime shifts. SR 11-7 and equivalent model risk management guidance address the first three regions adequately. The fourth is where standard validation methodology fails, and where an external adversarial layer is not an addition. It is a requirement.
The five gates
Each gate uses the statistical framework appropriate to its question.
Modular validation methodology, as Joseph Simonian framed it in the Journal of Financial Data Science in 2020. Each gate below cites the peer-reviewed methodology it operationalizes. Every threshold has a stated rationale. Every deviation from the standard method is flagged in the verdict-form.
Detects strategies whose observed edge does not survive correction for multiple testing, non-normality of returns, and length of the observation window.
Detects strategies whose profitability is concentrated in a single market state and would deteriorate under an unencountered regime.
Detects strategies whose significance was inflated by directed exploration during development, the failure mode that machine-learning workflows produce by default.
Detects strategies whose stated confidence estimates diverge from realized outcomes. A signal that is directionally correct but miscalibrated breaks position sizing.
Detects strategies whose expected edge is positive but whose variance is large enough that the strategy may not remain profitable under realistic path variations.
Deployment vocabulary
Model Trust Levels. From MTL-1 Untested to MTL-5 Live.
A shared vocabulary for how much trust to place in a probabilistic prediction system. Numbered ascending intensity. Each level defined by empirical passage of specific gates. Strict nesting. Binary per level. Publicly contestable. Time-bounded.
Every MTL attribution is publicly contestable through the audit trail supplied with the verdict-form. Any independent third party may reproduce the computation and challenge the attribution.
What allocators receive
A verdict-form structured as a medical report.
Findings first, methodology available on request, references cited for audit. Two layers: the upper layer states the verdict in institutional vocabulary; the lower layer supplies the technical evidence. The upper layer never overstates what the lower supports.
Verdict scope statement
Regime scope, sizing envelope, portfolio-construction assumptions. Deployment outside the envelope invalidates the verdict. A verdict on a signal is not a verdict on the portfolio built from that signal.
Triangulation panel
Risk category times business context times regulatory frame. Adapted from the three-lens framework Senthil Kumar established while Chief Risk Officer at BNY Mellon. A verdict against one triangulation cell does not extend to another.
Communication layer
Upper allocator vocabulary (shortfall, drawdown, liquidity, embarrassment, per Matt Bank's Four Horsemen framework at GEM). Lower technical evidence (DSR, BSS, ECE, bootstrap CIs). Both required.
Citation guidance
The verdict-form is designed to be citable by model owners, independent validators, and internal audit across all three lines of institutional defense. Framework, not model, following Roland Stamm at Acadia.
Reproducibility appendix
Doctrine version applied, ʼCɩcɛ validator commit hash, statistical package versions, random seeds, bootstrap block sizes, HMM state counts, data provenance. Anyone can re-run.
Style drift trigger
Any material change in features, instrument universe, risk parameters, or claimed problem domain invalidates the previous verdict. Re-validation required, not optional.
Input integrity
Sequence commitment.
Nyalai never requests permission to publish a case, or to use it in any downstream decision, before a verdict is rendered. Consent for publication or downstream reference is collected after the verdict is rendered, on a document the submitter reviews. Refusal of downstream use does not alter the verdict itself.
1. Submission is frozen first. Timestamp and hash are recorded before any conversation about downstream use begins.
2. Gates run without stake exposure. The strategy author is never told, during evaluation, what downstream consequences might attach to the verdict.
3. Consent comes after the verdict. Publication, private client disclosure, aggregate research, or external reference are all conditioned on consent collected after the verdict is fixed.
This is not a mental-state promise. It is an order-of-operations commitment that Nyalai controls and any third party can audit. Per Refusal Doctrine section 12.4. A discipline of validity, not virtue.
The full methodology is public.
Every gate, every threshold, every methodological choice is documented in the Refusal Doctrine v0.3.3.1. Twelve sections plus acknowledgments plus primary-source bibliography. Published under CC0 1.0. Anyone can reimplement it, contest it, or diverge from it.