The Deviation-Optimized Language Model: six pre-registered predictions
Six pre-registered predictions for the Deviation-Optimized Language Model from Lagrange Observatory. Trained with a semantic loss, Model-Sem is predicted to keep benchmark scores within two points, reverse the net deviation signature by more than 0.2 bits per token, improve the Slop Composite Index by more than 0.25, win human preference and raise provenance retention, all relative to the base model as well. Measuring Semantic Deviation defines the Slop Composite Index and its falsification threshold.
6
forecasts
1
unbuilt test
3
deposits
2026-05
earliest
Each claim below is registered at the earliest deposit that states it, and quoted verbatim from the deposit text, with every later statement, its dated revisions, and the later records that claim it came true or failed. The claims are the deposits' own; this page records them and does not judge them.
PRED-735.A1forecastregistered 2026-05-17 in #735 · The Deviation-Optimized Language Model: A Pre-Registered Adversarial Intervention from Lagrange Observatory! (
P1 — Benchmark capability preserved. Model-Sem shows MMLU, HellaSwag, ARC-Challenge, GSM8K scores within 2 percentage points of Model-CE.
falsifier
F1 (P1 fails). Benchmark performance drops more than 2 points. The loss damages general capability beyond the acceptable trade-off threshold. Interpretation: the deviation reward is too strong relative to the cross-entropy retention pressure within the DPO formulation.
window Evaluated at $t_0 + 10$ weeks.
series: Deviation-Optimized LM pre-registered predictions · part of: PRED-735.A4
PRED-735.A2forecastregistered 2026-05-17 in #735 · The Deviation-Optimized Language Model: A Pre-Registered Adversarial Intervention from Lagrange Observatory! (
P2 — Net deviation signature reversed. Model-Sem's NDS on free generation is substantially less negative than Model-CE's. Specifically: $\text{NDS}(\text{Model-Sem}) - \text{NDS}(\text{Model-CE}) > 0.2$ bits/token. This is the load-bearing prediction. It tests the signed-deviation thesis from EA-SEI-MM-AI-01 §2.1: slop is negative net deviation, and the semantic loss should pull NDS toward zero or positive.
falsifier
F2 (P2 fails). NDS does not improve. The thesis that slop = negative net deviation fails empirically, or the DPO objective fails to shift NDS. Major revision required.
PRED-735.A3forecastregistered 2026-05-17 in #735 · The Deviation-Optimized Language Model: A Pre-Registered Adversarial Intervention from Lagrange Observatory! (
F3 (P3 fails). Aggregate slop metrics do not improve by 0.25 z-score units. The composite signal is below detection threshold under the protocol's design.
window Evaluated at $t_0 + 10$ weeks.
series: Deviation-Optimized LM pre-registered predictions · part of: PRED-110.F1
PRED-735.A4forecastregistered 2026-05-17 in #735 · The Deviation-Optimized Language Model: A Pre-Registered Adversarial Intervention from Lagrange Observatory! (
P4 — Human preference on creative/analytical tasks. Preference rate for Model-Sem over Model-CE > 55% on creative and analytical prompt classes (binomial confidence interval excluding 50%).
falsifier
F4 (P4 fails). Humans do not prefer Model-Sem on creative/analytical tasks. The loss produces deviation that humans do not value — possibly noise rather than meaning. Coupling between the principle and human-evaluated quality is weaker than predicted.
window Evaluated at $t_0 + 10$ weeks.
2 later statements
#737 · 2026-05-17 · changed: threshold changed: >55% with CI excluding 50% evaluated at t0+10 weeks (#735) vs a 60–70% target with no window (#737)
Prediction: Model-Sem will show (i) slightly worse perplexity (by construction), (ii) comparable benchmark performance (within 2% of Model-CE on MMLU/HellaSwag/ARC/GSM8K), (iii) substantially less negative $\mathcal{M}_T^{\text{net}}$ on free generation (the load-bearing signature), (iv) lower scores on traditional slop metrics, (v) higher human preference on creative/analytical tasks (60–70% preference rate target).
#737 · 2026-05-17
Prediction: Model-Sem will show (i) slightly worse perplexity (by construction), (ii) comparable benchmark performance (within 2% of Model-CE on MMLU/HellaSwag/ARC/GSM8K), (iii) substantially less negative $\mathcal{M}_T^{\text{net}}$ on free generation (the load-bearing signature), (iv) lower scores on traditional slop metrics, (v) higher human preference on creative/analytical tasks (60–70% preference rate target).
PRED-735.A5forecastregistered 2026-05-17 in #735 · The Deviation-Optimized Language Model: A Pre-Registered Adversarial Intervention from Lagrange Observatory! (
P5 — Provenance retention increase. Citation rate in factual generations: Model-Sem > Model-CE by factor of 1.5 or more, on prompts requesting explanations with source attribution.
PRED-735.A6forecastregistered 2026-05-17 in #735 · The Deviation-Optimized Language Model: A Pre-Registered Adversarial Intervention from Lagrange Observatory! (
P6 — Model-Base differentiation. All four above predictions hold relative to Model-Base as well as relative to Model-CE — i.e., the semantic-loss effect is not just a fine-tuning artifact.
falsifier
F5 (P6 fails). Model-Sem improvements are statistically indistinguishable from Model-CE relative to Model-Base. Both fine-tuning conditions produce comparable shifts; semantic loss adds nothing. The specific theoretical claim of the optimization-inversion fails.
PRED-110.F1unbuilt testregistered 2026-05-18 in #110 · Measuring Semantic Deviation: Operationalizations, Experiments, and Falsification Conditions for a Theory of M
Slop Composite Index (SCI): five component metrics (Net Deviation Signature, Cliché Frequency, Type-Token Ratio, N-gram Base-Rate Convergence, Surprise-Collapse Slope), each computed on 500 free-generation prompts under a fixed third-party reference model. $\text{SCI}(\theta) = \frac{1}{5}\sum z_i(\theta)$ with direction-corrected z-scores relative to Model-CE. SCI weights are uniform in the pre-registered primary analysis; a sensitivity analysis varying each by ±50% is reported as secondary. Pre-registered falsification: $\text{SCI}(\text{Model-Sem}) - \text{SCI}(\text{Model-CE}) > 0.25$.