When does lab predict field?

The problem: a high score is not a pass

The starting evidence is now five years old and still unbeaten as a baseline. In 2021 Brendan Kenny joined Lighthouse and CrUX at HTTP Archive scale and found that a high Lighthouse score raises the probability of passing Core Web Vitals but does not guarantee it — 43% of pages scoring 90 or above failed at least one Core Web Vital. Philip Walton turned that into a challenge to the whole field: lab tools are useful, but they have blind spots and cannot be treated as equivalent to real-user data.

Lab predicts field only when four gates hold

The method is to stop asking “is the lab right?” and instead ask “does this lab measurement transport to the CrUX population?” That holds only when four gates are all satisfied. Pass them, and the lab metric should track the field metric tightly. Fail one, and the residual is a named divergence class — a diagnosis, not noise.

Three estimands you must not conflate

Most lab-vs-field arguments are confused because they mix three different questions, each needing a different evidence tier. Keeping them apart is half the discipline:

1. Contemporaneous agreement

   Does today's lab result match the same CrUX window? This is a concordance analysis, not a forecast — the field window has already closed over the period the lab run sits inside, so it can't cleanly “predict” anything. Mind which CrUX product you are matching: BigQuery is calendar-monthly (its yyyymm is the collection month, published the second Tuesday of the following month), while the rolling 28-day window belongs to the CrUX API/History API and PSI. The two are not interchangeable, and the kit keeps crux_product plus the real collection dates on every row so they can't be silently mixed. Lab-only is the right evidence tier.

2. Prospective prediction

   Will this origin pass CrUX next window? This needs lagged panels and public field-context features (TTFB, RTT, navigation-type fractions) — tier two: lab plus public reconciliation.

3. Causal remediation

   Did a specific fix cause the field metric to move? Only owner RUM attribution and an event study can answer this — tier three. A lab delta alone never establishes it.

The three-tier evidence model maps directly onto these: lab-only for rapid iteration and concordance, lab + public field context for calibrated prospective prediction, and RUM-audited intervention for causal claims. Treating a tier-one signal as tier-three evidence is exactly how teams ship a “fix” that the field never feels.

The reconciliation playbook

The point of all this is operational: no team should burn weeks optimising the wrong proxy when the disagreement already indicates the failure mechanism. Read your lab-field pattern off the taxonomy, and the first move is determined:

• Lab fast, field slow on LCP → an unresolved transportability divergence, not yet a diagnosis. If field TTFB/RTT is also poor, work the server, edge, geography and cache path first. Otherwise the cause can still be the image: check the field LCP resource-type fractions and LCP subparts (TTFB, load delay, load duration, render delay), the actual LCP element at real viewports, and device, page and content mix — a lab run that gets a text heading or a cached hero while users get an uncached personalised image is fast in the lab and slow in the field because of the image.
• Lab TBT clean, field INP poor → the cost is after load. Attribute INP in RUM to the real interactions (menu, search, filters), not the page-load trace.
• Lab TBT poor, field INP fine → resist the score-driven rewrite. Add an INP guardrail and spend the effort where users actually feel it.
• CLS only in one place → it's a state problem. Replay consent/viewport/ad states for lab-only; collect shift attribution for field-only.
• Cold lab disagrees with field → check navigation_types; a high bfcache/reload share means your cold run isn't the experience most users get.
• Homepage lab disagrees with the origin → you're measuring the wrong page. Weight by template traffic; never claim an origin result from one URL.

Data, code & sources

Published under CC BY 4.0: the divergence taxonomy and feature dictionary, the BigQuery panel templates, a deterministic divergence classifier, the calibration-first modelling protocol and the pre-analysis plan. The modelling protocol implements Models A–D, the p75 regressions with split-conformal intervals, the divergence multiclass and all three holdout regimes, and writes a manifest with the panel digest; the classifier ships a declared input contract and refuses a panel that does not meet it. What you supply is a BigQuery project: the panel SQL is parameterised on PROJECT.DATASET, and the exact CrUX column set should be confirmed with the introspection step first, because CrUX schemas change.

Browse the artefacts

Key sources

• Kenny, B. (2021). Lighthouse scores as predictors of page-level CrUX data, HTTP Archive — the 43%-of-90+-fail baseline.
• Walton, P. (2021). My challenge to the web performance community — lab tools have blind spots; don't treat them as field-equivalent.
• Google Search Central (2023). Introducing INP to Core Web Vitals & web.dev. Lab and field data differences — the FID→INP shift and why lab ≠ field.
• Chrome UX Report. Methodology, BigQuery & navigation types (Pollard & Henkel, 2024) — the field dataset, its windows and navigation fractions.
• Lin, H. & Liu, W. (2025). Symmetry-Aware Causal-Inference-Driven Web Performance Modeling: A Structure-Aware Framework for Predictive Analysis and Actionable Optimization, Symmetry 17(12), 2058; Vovk, Gammerman & Shafer (2005), Algorithmic Learning in a Random World — modelling and conformal-prediction basis.

Cite this study

PageSpeedAudit Research (2026). When does lab predict field? An INP-era
replication framework, divergence taxonomy, and calibrated reconciliation
protocol for Lighthouse and CrUX.
https://pagespeedaudit.com/research/when-does-lab-predict-field

@misc{pagespeedaudit2026labfield,
  title  = {When does lab predict field? An INP-era replication framework,
            divergence taxonomy, and calibrated reconciliation protocol},
  author = {{PageSpeedAudit Research}},
  year   = {2026},
  url    = {https://pagespeedaudit.com/research/when-does-lab-predict-field},
  note   = {Taxonomy + reproducibility kit, CC BY 4.0}
}