Semantic deviation and Framework 15: pre-registered tests of model convergence
These claims pre-register tests of whether machine output converges and RLHF compresses deviation. The Semantic Deviation Research Program and Measuring Semantic Deviation predict negative signed deviation for AI text, a base-versus-instruct differential, stability across scale and agreement across judges. Framework 15 and its papers from Lagrange Observatory! freeze the 90-day retrieval-basin winding-number protocol and the deviation-optimized DPO experiment, with results deposits owed.
5
forecasts
3
standing claims
8
unbuilt tests
7
deposits
2026-04
earliest
Each claim below is registered at the earliest deposit that states it, and quoted verbatim from the deposit text, with every later statement, its dated revisions, and the later records that claim it came true or failed. The claims are the deposits' own; this page records them and does not judge them.
PRED-686.B2forecastregistered 2026-04-24 in #686 · Mycelial Cardboard Boxes: Market Analysis and Production Strategy
The critical experiment is whether colonized cardboard survives folding — test this immediately with a small batch. If the mycelium heals across fold lines (likely, if folded while still actively growing), the product ships flat, stores efficiently, and the customer unfolds their own living box. That's a better experience than receiving a pre-formed box, and it's a better story.
PRED-107.F3unbuilt testregistered 2026-05-17 in #107 · Audited Claims for the Semantic Deviation Research Program — The Glas Function: An External-Format Restatement
The right experimental design is component-decomposed. Conventions:
- Model-Base is the pre-fine-tune checkpoint of the chosen open-weight model (e.g., the base Llama-3.1-8B prior to any post-training).
1 later statement
#107 · 2026-05-17
| Model-Full (= Model-Sem) | on | on | on | on | on |
The single-component conditions (Model-π, Model-Dev, Model-Coh) isolate the uplift attributable to each signal. The full condition (Model-Full) replicates the design of the corpus's existing DPO protocol. The differences between conditions are the scientifically interesting quantities, not the absolute performance of the full condition.
series: Semantic Deviation Research Program audited claims
PRED-107.F4unbuilt testregistered 2026-05-17 in #107 · Audited Claims for the Semantic Deviation Research Program — The Glas Function: An External-Format Restatement
Black-box judge replacement test. A scientifically essential robustness check, directly addressing the reward-model overoptimization concerns characterized in Gao et al. 2023: train Model-Sem against Judge-A; evaluate against Judge-B (a different frozen model with different weights). If Model-Sem's uplift persists against Judge-B, the result generalizes beyond the specific judge. If it disappears, the result is judge-specific reward hacking. This test is cheap (one additional evaluation pass) and load-bearing.
series: Semantic Deviation Research Program audited claims · related: PRED-107.F9
PRED-107.G1forecastregistered 2026-05-17 in #107 · Audited Claims for the Semantic Deviation Research Program — The Glas Function: An External-Format Restatement
P2 — Pre-RLHF vs. post-RLHF deviation differential. Base models (Llama-3.1-8B base versus Llama-3.1-8B-Instruct) should exhibit different mean $\bar{\delta}$ on matched prompts. The framework predicts the chat-tuned model exhibits lower mean signed deviation — that RLHF produces convergence pressure measurable in the signed-deviation statistic. Same two-sided Mann-Whitney U at $\alpha = 0.05$.
1 later statement
#110 · 2026-05-18P2 (RLHF flattening). Post-RLHF chat-tuned models exhibit lower $\bar{\delta}$ than their pre-RLHF base counterparts on matched prompts — consistent with Ouyang et al.'s (2022) observation that RLHF produces more uniform outputs. Tested on meta-llama/Llama-3.1-8B (base) vs. meta-llama/Llama-3.1-8B-Instruct (chat-tuned), evaluated on 100 prompts from the OpenAssistant dataset matched by length bin. Limited to open-weight models where base weights are available.
series: Semantic Deviation Research Program audited claims · register: I:063, I:065
PRED-107.G2standing claimregistered 2026-05-17 in #107 · Audited Claims for the Semantic Deviation Research Program — The Glas Function: An External-Format Restatement
A reasonable advance prediction, based on the structure of the framework and the existing alignment literature: the provenance component will carry more uplift independently than the deviation component.
falsifier
This is a falsifiable prediction. If Model-Dev outperforms Model-π in human preference ratings on a matched evaluation set, the deviation hypothesis is vindicated independently. If Model-π outperforms Model-Dev, the framework retains value but its center of gravity shifts: provenance-aware preference optimization becomes the durable contribution, while deviation becomes a secondary stabilizer.
PRED-107.G4standing claimregistered 2026-05-17 in #107 · Audited Claims for the Semantic Deviation Research Program — The Glas Function: An External-Format Restatement
P3 — Effect size scaling. The Slop vs. Human deviation differential should be stable or grow with model scale (computed across multiple model sizes within a family — e.g., Llama-3.1-8B-Instruct, 70B-Instruct, 405B-Instruct). If the differential disappears at scale, the framework's predictions are small-model artifacts.
series: Semantic Deviation Research Program audited claims
PRED-107.G5forecastregistered 2026-05-17 in #107 · Audited Claims for the Semantic Deviation Research Program — The Glas Function: An External-Format Restatement
P4 — Cross-judge consistency. The differential should replicate when computed against a different reference model. Test the same Slop/Human corpus against Mistral-7B-Instruct's logits; the Spearman rank correlation between per-output $\bar{\delta}$ rankings under Llama and Mistral should exceed 0.7. If it does not, the deviation statistic is judge-specific, and a much more careful argument is required to claim it measures anything intrinsic to the texts.
1 later statement
#110 · 2026-05-18P3 (Cross-judge consistency). The differential replicates under mistralai/Mistral-7B-Instruct-v0.3. Spearman rank correlation between per-output $\bar{\delta}$ rankings under Llama and Mistral exceeds 0.7. Failure indicates judge-specificity, not an intrinsic text property.
series: Semantic Deviation Research Program audited claims
PRED-108.F1unbuilt testregistered 2026-05-17 in #108 · Framework 15 — Measurement of Meaning: Operations from Lagrange Observatory! within the Semantic Physics Matri
A pre-registered field protocol. The retrieval-basin 90-day winding-number measurement (Framework 15 Paper 03) is a real instrument with API specifications, frozen extractors, three-condition controls, and falsification thresholds.- A pre-registered ML intervention. The deviation-optimized DPO experiment (Framework 15 Paper 04) is a real experiment with judge-model specification, adversarial validation, honest budget, and falsification thresholds.
PRED-108.G1standing claimregistered 2026-05-17 in #108 · Framework 15 — Measurement of Meaning: Operations from Lagrange Observatory! within the Semantic Physics Matri
If the retrieval-basin protocol shows that inscription does not deform AI surfaces in the predicted way, the principle's retrieval-basin claim requires Sharks-level revision. If the deviation-optimized DPO produces models with positive net deviation and improved human preference, the discipline's training-objective claims are vindicated empirically. Framework 15 supplies the evidence-stream the prior fourteen frameworks called for but did not themselves produce.
PRED-735.F1unbuilt testregistered 2026-05-17 in #735 · The Deviation-Optimized Language Model: A Pre-Registered Adversarial Intervention from Lagrange Observatory! (
Status. Pre-registered protocol specification. The 10-week experiment's Day 0 begins at deposit. No results are reported here; predictions, falsification conditions, and hyperparameters are frozen at deposit time. Results paper EA-SEI-MM-AI-02-RESULTS will be deposited at $t_0$ + 10 weeks. Any deviation from the protocol during execution is documented as a protocol amendment.
PRED-736.F1unbuilt testregistered 2026-05-17 in #736 · Measuring Meaning in Retrieval Basins: A 90-Day Winding-Number Protocol from Lagrange Observatory! (EA-SEI-MM-
Status. Pre-registered protocol specification. Day 0 begins at deposit. The paper contains no findings. Empirical results will be deposited as EA-SEI-MM-02-RESULTS at $t_3 + 7$ days. Predictions, falsification conditions, query sets, and code structure are frozen at deposit time; deviations from the protocol during execution will be reported as protocol amendments rather than retroactive design choices.
PRED-736.F2unbuilt testregistered 2026-05-17 in #736 · Measuring Meaning in Retrieval Basins: A 90-Day Winding-Number Protocol from Lagrange Observatory! (EA-SEI-MM-
The three representations should produce correlated (Pearson $r > 0.7$) but not necessarily identical magnitudes. If $r < 0.5$ between any pair, the protocol is paused and re-calibrated before $t_3$.
PRED-737.B1forecastregistered 2026-05-17 in #737 · The AI System as Closed-System Test Bed: Operations from Lagrange Observatory! on the Inference-Time Forward P
They are intended to be deposited within weeks of each other, with code and data, as a unified research module.
window within weeks of each other
PRED-737.F1unbuilt testregistered 2026-05-17 in #737 · The AI System as Closed-System Test Bed: Operations from Lagrange Observatory! on the Inference-Time Forward P
Status. Pre-registered protocol specification. The May 17, 2026 deposit contains no experimental findings. Each test specified herein has a Day 0 that begins at deposit. Results papers will be deposited separately as EA-SEI-MM-AI-01-RESULTS (Tests 1–2; Glas timeline ~6 weeks) and EA-SEI-MM-AI-02-RESULTS (Test 3, the deviation-optimized training experiment, specified in the companion paper).
PRED-110.A1forecastregistered 2026-05-18 in #110 · Measuring Semantic Deviation: Operationalizations, Experiments, and Falsification Conditions for a Theory of M
P1 (Machine-output convergence). AI-generated text in matched human/AI corpora exhibits statistically significant negative mean $\bar{\delta}$ relative to matched human text. Two-sided Mann-Whitney U at $\alpha = 0.05$, minimum effect size Cohen's $d > 0.5$. A positive result motivates a second-stage test on human-labeled low-quality AI text to determine whether the effect sharpens in the slop regime.
falsifier
P1 failure disconfirms this paper's first high-stakes prediction — that benchmark AI text exhibits a negative signed-deviation signature. It would block the proposed training intervention in its current form while leaving the broader field-deformation measurement program open. P1 success with P3 failure retreats to a weaker, judge-relative claim. P1 and P3 success warrants the training intervention.
window This week (roadmap: cheapest dangerous test)
register: I:184
PRED-110.F2unbuilt testregistered 2026-05-18 in #110 · Measuring Semantic Deviation: Operationalizations, Experiments, and Falsification Conditions for a Theory of M
Horizon
Milestone
Budget
This week
Cheapest dangerous test (P1–P3) on GPT-wiki-intro + HC3, ~1 A100-hour
$50–$100
This month
Operationalization-stability: 50 texts evaluated under F1 across 3 reference models, cross-model $\bar{\delta}$ rank-correlation reported
$200–$500
This quarter
F2 protocol day-0 launch; 90-day window; 30 queries × 4 intervals × 6 surfaces
$1,500–$3,000
window This week / This month / This quarter / This year