Lunit · Workshop paper under review, 2026
TL;DR. Gate-space metrics tell us how much routing mass moved, not which expert-output directions received it. We define an exact conditional reference that preserves the captured expert outputs, expert union, and intact before/after gate pairs while relabeling their correspondence. Its centered ratio factors as spectral capacity times directional utilization. Most layers lie below the reference mean, but a few aligned layers dominate checkpoint averages. A preregistered geometry-based allocator then reverses across checkpoints. The geometry is a fast diagnostic, not a model-agnostic shortcut to behavioral importance.
Suppose two experts receive equal and opposite gate changes: one gains mass a, the other loses a. A router-only metric sees the same displacement in every case. The output perturbation, however, depends on the expert vectors receiving that displacement.
Switch the expert configuration; the gate movement stays fixed.
The experiment replays one generated continuation under query-only and gold-conditioned contexts. At each token and layer, it captures the student expert outputs and the teacher/student gate pairs over their observed expert union. The reference then relabels the intact gate pairs across those fixed expert-output columns.
Keep the generated tokens, captured student expert outputs, expert union, sparsity, overlap, total mass, and intact before/after gate pairs.
Uniformly permute which expert-output column receives each intact gate pair. The resulting assignments need not be executable routes.
An analytic second moment gives the exact conditional reference in O(dm) time after capture, without Monte Carlo reruns.
The centered correspondence ratio compares the observed gate-to-expert alignment with the mean alignment over that relabeling orbit. A value below one means below the conditional mean; it does not mean rare, harmless, or behaviorally irrelevant.
The centered ratio factors exactly into two interpretable terms. Spectral capacity describes how strongly the captured expert outputs could amplify a centered gate direction. Directional utilization measures how much of that available capacity the observed movement actually uses.
DeepSeek makes the sparse-tail problem vivid. Its equal-layer mean is 1.374 even though 23 of 26 layers lie below one. A few aligned layers dominate the checkpoint summary. The factorization explains why: capacity amplifies only when the realized gate direction uses it.
A useful diagnostic does not automatically make a useful allocator. We froze a rule that selected the layer with maximum centered correspondence ratio and compared it with a high-capacity, low-ratio control. Selection used 200 prompts; evaluation used a disjoint 400-prompt pool and a complete 42-layer census across OLMoE and DeepSeek.
The failure mechanism replicates on a disjoint MATH census. Capacity consistently enlarges the relabeling reference and correlates negatively with held-out lift, while utilization correlates positively. Their product is checkpoint-unstable. Maximizing utilization alone also selects no oracle. The factorization diagnoses the reversal; it does not rescue zero-shot allocation.
The exact orbit answers where moved gate mass lands. It does not provide a checkpoint-independent ranking of which layers matter behaviorally.
A more promising signal appears within the same checkpoint across related tasks. Full per-layer behavioral profiles correlate across GSM8K and MATH at Spearman 0.70 for OLMoE and 0.85 for DeepSeek. Using a costed prior-task census to select a target-task layer gets close to the target-task oracle without any target-task selector forwards.
This is descriptive evidence from two tasks and two anchor checkpoints, not a universal transfer theorem. It nevertheless changes the engineering hypothesis: amortize one checkpoint-level behavioral calibration across related tasks, then verify the transferred candidates.
Use overlap, variation, or related gate statistics to locate where routing changed, without treating them as functional evidence.
Once expert outputs are captured, use the correspondence reference to explain whether moved mass aligns unusually with available directions.
Acquire a behavioral layer profile when the cost is justified, transfer a shortlist only within a calibrated checkpoint, and test finalists directly.
Geometry narrows and explains the search. Behavioral calibration decides where consequence actually lies.
Relabeled assignments are a conditional attribution reference and need not be executable routes. The orbit does not test whether the original expert union was well chosen. The behavioral allocation study covers two anchor checkpoints, one primary task, one perturbation family, and a descriptive two-task transfer analysis. It is not causal mediation evidence or a deployment guarantee, and a new checkpoint still requires calibration.
Status note. This page describes a workshop paper under review. Venue and paper links will be updated after the review process permits a stable public reference.
@unpublished{caruzzo2026expertdirections,
title={Which Expert Directions Receive Moved Gate Mass? An Exact Correspondence Reference for MoE Routing Changes},
author={Caruzzo, Cedric and Yoo, Donggeun and Kim, Tae Soo},
note={Workshop paper under review},
year={2026}
}