Run 2026-07-28. This is the gate phase: everything downstream (storage design, the harvester, every domain backtest) depends on the answer here.
FactIQ stores and serves only the current, latest-revised value for every observation. It does not retain the originally-published print, does not expose any way to request "value as known on date D," and has no column, dimension, or parameter anywhere in its schema that distinguishes a vintage from the current best estimate. This was true for every single period tested, with no exceptions, across two unrelated data providers (BLS, BEA).
Practical consequence: any backtest built directly on FactIQ historical data is backtesting against information that did not exist at the time — it is testing hindsight, not a strategy that could have been run live. This is exactly the failure mode this project exists to catch.
Confirmed via information_schema.columns against the shared series,
data_points, and dimensions tables in two unrelated schemas (bls and
china — deliberately picked from different providers to rule out a
one-off gap):
data_points: series_id, time, value, updated_at — no vintage,
revision, publication_date, release_date, or ingestion_timestamp column.series: id, dataset_code, series_id, series_title,
human_friendly_title, human_friendly_description, measurement_units,
frequency, adjusted_for_seasonality, state, county_or_metro_area,
begin_time, end_time, footnote_codes, updated_at, data_type,
tabular_columns — same story, updated_at is the only date-like field
beyond the observation time itself.dimensions (checked china's distinct dimension_type values):
frequency, hierarchy_level_0..3, unit — no base-year or vintage
dimension type.data_points.updated_at looked like it might be a usable proxy — it
turned out not to be. See Test 3.
Method: pulled current FactIQ values via run_sql for nonfarm payrolls
(bls.CES0000000001, FRED counterpart PAYEMS) and real GDP % change
(bea.nipa|A191RL|T10301, FRED counterpart A191RL1Q225SBEA), then queried
every historical vintage ALFRED has ever recorded for those same
observation dates. Comparison and diffing are done entirely in
src/validation/vintage_probe.py (re-run with python -m
src.validation.vintage_probe) — no hand-typed numbers, no LLM arithmetic.
Nonfarm Payrolls (thousands of persons):
| Period | FactIQ value | Original print (ALFRED, date) | Current revised (ALFRED) | # vintages | Verdict |
|---|---|---|---|---|---|
| 2025-11 | 158,449 | 159,552 (2025-12-16) | 158,449 | 3 | matches CURRENT |
| 2025-12 | 158,432 | 159,526 (2026-01-09) | 158,432 | 3 | matches CURRENT |
| 2026-01 | 158,592 | 158,627 (2026-02-11) | 158,592 | 3 | matches CURRENT |
| 2026-02 | 158,436 | 158,466 (2026-03-06) | 158,436 | 3 | matches CURRENT |
| 2026-03 | 158,650 | 158,637 (2026-04-03) | 158,650 | 3 | matches CURRENT |
The November 2025 row is the clearest single data point in this whole survey: the originally published payrolls figure was 159,552,000. As of today FactIQ reports 158,449,000 for that same month — a 1,103,000 person gap, and FactIQ shows only the revised number, with zero indication that a different figure was ever published.
Real GDP, % change annualized:
| Period | FactIQ value | Original print (ALFRED, date) | Current revised (ALFRED) | # vintages | Verdict |
|---|---|---|---|---|---|
| 2025 Q1 | -0.6 | -0.3 (2025-04-30) | -0.6 | 4 | matches CURRENT |
| 2025 Q2 | 3.8 | 3.0 (2025-07-30) | 3.8 | 3 | matches CURRENT |
| 2025 Q3 | 4.4 | 4.3 (2025-12-23) | 4.4 | 2 | matches CURRENT |
| 2025 Q4 | 0.5 | 1.4 (2026-02-20) | 0.5 | 3 | matches CURRENT |
| 2026 Q1 | 2.1 | 2.0 (2026-04-30) | 2.1 | 3 | matches CURRENT |
2025 Q4 is the standout: the advance estimate was +1.4%, the currently-revised figure is +0.5% — a full percentage point of swing, enough on its own to flip a directional call on growth momentum. FactIQ shows only 0.5%.
Every single row in both tables matches the current revised value and none match the original print. Ten out of ten. This is not a partial or inconsistent result — it's a clean, unambiguous pattern.
Raw output: findings/vintage-probe-results.json.
Tried every plausible mechanism:
data_points.updated_at as a proxy for "value as it stood before date
D": queried WHERE updated_at <= '2025-12-01' AND time = '2025-11-01'
for nonfarm payrolls — zero rows returned. The single row for that
observation carries updated_at = 2026-07-02; there is no earlier row
holding the original 159,552 value. This confirms the update is
destructive (overwrite-in-place), not append — the prior value is gone,
not superseded by a newer row that could be filtered out.run_sql, get_series, or describe_dataset
accepts an as-of date.Conclusion: there is no way, today, to ask FactIQ for a historical value as it was known on a past date. The information required to answer that question has already been discarded.
Time-boxed given how unambiguous Tests 1–3 already are. Two observations, neither a full worked example of silent restatement (none was found or ruled out conclusively — flagged as open, not resolved):
dimensions table has no base-year or vintage dimension type at
all (Test 1), which is structurally consistent with not preserving an
original basis when a series is rebased — but this wasn't proven with a
concrete before/after example.china_customs carries two dataset codes: china_customs (HS-level
detail) and china_customs_prelim (aggregate preliminary totals, e.g.
china_customs_prelim_total_export_cny). This looked initially like it
might be a paired preliminary/final vintage capture, but no non-prelim
counterpart series exists for the same aggregate concept — it appears to
be China's own separately-published preliminary release, not a
FactIQ-side vintage mechanism. Not chased further; worth a direct
question if raised.china_customs_prelim_total_export_cny has three consecutive
months (2025-06, 2025-07, 2025-09) carrying the identical value
23445.46350339 — looks like a stale carried-forward figure. Not
independently verified against the Chinese-language source; flagged as a
data-quality caveat, not a vintage finding.Baseline snapshot taken today across three series spanning three unrelated
providers (BLS payrolls, BEA GDP, China Customs preliminary exports) —
findings/drift-baseline.json. Diff logic (src/validation/drift_check.py,
diff_snapshots) is written and deterministic. findings/drift-log.md is
initialized and will accumulate entries.
This is not yet running on a schedule. FactIQ has no plain REST endpoint in this project — it's reachable only via MCP tool calls inside a Claude Code session. The Phase 3 harvester needs its own answer for how a standalone daily cron process authenticates to FactIQ before Test 5 can run unattended; until then, snapshots have to be re-pulled manually. This is a real architectural open question, not a detail — flagging it now so it doesn't get silently assumed away when Phase 3 starts.
See findings/questions-for-factiq-vintage.md (already sent). Given what
Test 2 shows, the highest-priority follow-ups from that list are A2/A3
(is the update in-place, and is any pre-revision value retained anywhere
internally) and C7/C8 (does FactIQ pull from the advance release or a
batch/bulk file that already reflects revisions) — everything else in that
list is now secondary to confirming those.
FactIQ replied to findings/questions-for-factiq-vintage.md directly, in
writing. This section records what they said, verbatim where it matters,
against the specific tests above — this is now a source-confirmed finding,
not just an empirically inferred one.
We currently do not save pre-revision data and don't have point-in-time vintages of the data. It's not on our near-term (1 month) roadmap, but I will certainly look into it.
Mapped to the open questions this project raised:
updated_at resets on every refresh
regardless of whether the value actually changed, which rules it out even
as a weak "did this change recently" signal.harvest_date remains the only point-in-time mechanism this project has
or can plan around, and re-running the vintage probe periodically remains
the correct way to notice if that ever changes, per
findings/status-summary.md.FactIQ also flagged newly-released satellite-derived data — night lights,
port activity, and fires. All three were already found present and load-
bearing in Phase 0 (findings/00-capability-survey.md's satellite table),
run one day before this survey's query date — night lights was in fact
already identified there as the substitute proxy for Domain A after the
Yunnan reservoir gap. No new capability is added by this reply; it's
corroboration that FactIQ considers these intentional, current products
worth highlighting, not an incidental or unsupported dataset.
harvest_date insurance-policy column
described in the original spec is not a nice-to-have, it is the only
mechanism by which this project will ever have point-in-time data for
anything sourced from FactIQ. Every FactIQ-derived observation is only
as "as-of" as the day it happened to be harvested.Password required.