numeraire.reference#

ReferenceResult registry — reproduction targets as first-class, tiered data records.

A ReferenceResult pins a published result — an exact paper, venue, table, and paper version — to an expected metric dict plus a per-metric tolerance band on a named dataset, tagged by a data-access tier.

The tier axis is a DATA-ACCESS REQUIREMENT, never a statement of importance or rank:

  • PUBLIC — public/redistributable (or synthetic) data; the case runs unconditionally, including in CI.

  • CREDENTIALED — data that is programmatically fetchable with the user’s own subscription credentials (e.g. CRSP/Compustat through a connector); the case self-skips when those credentials are absent.

  • RESTRICTED — data that anyone may obtain but that is non-redistributable, so it needs a self-obtained local copy (e.g. CC-BY-NC returns that may never be committed); the case self-skips when that local copy is absent.

Tiers never encode importance or rank — a reproduction that needs licensed or restricted data is a first-class citizen. The tier plus an optional available predicate let CI stay green on public data while the same case runs verbatim wherever the private data is present — the connector pattern, one code path, no forked assertions.

(Disambiguation: a “reference result” here is a pinned published number; it is unrelated to the “reference libraries” — ipca / linearmodels — mentioned elsewhere in the project.)

This module is core infrastructure, not a method — it is exempt from the boundary rule’s methods/adapters import ban (it lives in numeraire proper, not numeraire.core, and imports only the standard library). The registry is process-global and open: numeraire ships a couple of its own references and any downstream package (numeraire-zoo, numeraire-yourlab) registers its reproduction targets the same way, then a single parametrized test drives them all (reference_params()).

The tolerance philosophy is the framework’s: a reference asserts an invariant plus a headline scalar within a band, never bit-equality — bands absorb data-vintage revisions (French/GW live-data drift). ReferenceResult.check() enforces the band and rejects a non-finite computed value (an all-NaN false green).

numeraire.reference.PUBLIC = 'public'#

Public/redistributable or synthetic data — the case runs unconditionally, including in CI.

numeraire.reference.CREDENTIALED = 'credentialed'#

Programmatically fetchable with the user’s own subscription credentials — skipped when absent.

numeraire.reference.RESTRICTED = 'restricted'#

Non-redistributable data needing a self-obtained local copy (never committed); skipped absent.

numeraire.reference.DATA_TIERS: tuple[str, ...] = ('public', 'credentialed', 'restricted')#

The closed set of data-access tiers, from least to most access-restricted.

numeraire.reference.VERIFIED = 'verified'#

The headline scalar was reproduced within band against the pinned fixture.

numeraire.reference.VERIFIED_WITH_CAVEAT = 'reproduced-with-caveat'#

The economics/invariants reproduce but an exact figure is sensitive (documented in notes).

numeraire.reference.UNVERIFIED = 'UNVERIFIED'#

A target recorded but not yet reproduced — carried so the queue is visible.

class numeraire.reference.ReferenceResult(name: str, paper: str, venue: str, year: int, table: str, expected: ~collections.abc.Mapping[str, float], tolerance: ~collections.abc.Mapping[str, float] = <factory>, tier: str = 'public', paper_version: str = 'published', data: str = '', status: str = 'verified', notes: str = '', available: ~collections.abc.Callable[[], bool] | None = None)[source]#

Bases: object

A pinned, tiered reproduction target: a paper figure/table matched within a band.

expected maps a metric name to the paper’s value; tolerance maps a (subset of those) metric names to an absolute band — a metric absent from tolerance must match exactly (band 0.0, only sensible for integer counts). available is an optional zero-arg predicate: when it returns False the case is skipped (its data is out of reach on this machine). PUBLIC cases normally leave it None (always available).

is_available() → bool[source]#

Whether this case’s data is reachable here (True when no predicate is set).

check(computed: Mapping[str, float]) → None[source]#

Assert every expected metric appears in computed and lands within its band.

Raises AssertionError on the first missing metric, non-finite value (guards against an all-NaN false green), or out-of-band deviation. Extra keys in computed are ignored.

numeraire.reference.register_reference(case: ReferenceResult, *, overwrite: bool = False) → ReferenceResult[source]#

Register case under its name. Raises on a duplicate name unless overwrite.

Returns the case so a module can register-and-bind in one line (FF2015 = register_reference(ReferenceResult(...))).

numeraire.reference.get_reference(name: str) → ReferenceResult[source]#

Return the reference result registered under name.

numeraire.reference.references(*, tier: str | None = None, available_only: bool = False) → tuple[ReferenceResult, ...][source]#

Return registered cases, name-sorted, optionally filtered by tier / availability.

numeraire.reference.clear_references() → None[source]#

Drop all registered cases (test-isolation helper; not for production paths).

numeraire.reference.reference_params(*, tier: str | None = None) → list[ParameterSet][source]#

Return pytest.param entries for registered cases, ready for @pytest.mark.parametrize.

Each entry carries the ReferenceResult as its single argument and the case name as its id; a case whose data is unavailable (ReferenceResult.is_available() is False) carries a pytest.mark.skip so a credentialed / restricted target self-skips on a machine that lacks the data instead of failing. pytest is imported lazily so this module stays import-clean at runtime (the helper is only ever called from a test).

Usage:

import pytest
from numeraire.reference import reference_params

@pytest.mark.parametrize("case", reference_params())
def test_reference(case):
    case.check(compute_metrics(case))