Historical validation and benchmark records

These are retained records, not current pass counts or coverage guarantees. They preserve the versions, dates, source pins, counts and limitations recorded before the October 27 documentation reorganization. A record with no original execution date is not assigned an invented date. Earlier source layouts and commands may differ from the current tree. Use current checks for the consolidated source layout.

Published a13 validation (October 26, 2026; bd14e4dc)

The implementation published at bd14e4dc passed 60 distinct tests (38 installed API/graphless/effects/default/weight/component tests and 22 separate graph-engine regressions), with zero skips. Seven API-guide Python blocks executed. The regular wheel contained 36 payload files, matching installed bytes; 17 generated modules matched canonical sources. The whole-call pilot completed 20/20 trials. These are bounded implementation checks, not accuracy or coverage guarantees.

New documentation walkthrough checks are recorded in the maintained tutorial and this refresh report; the old records below remain historical.

Historical documentation validation (a9)

This section preserves the earlier 0.1.0a9 documentation-pass record. The new candidate’s build, weighting, and HTML checks are recorded separately below.

On 2026-10-25, checks used the existing Python 3.11.14 interpreter at /Users/yonghanjung/.venvs/causalpy-py311/bin/python. The system python3 did not have the required numerical dependencies. No dependencies were added.

An existing installation target at /tmp/causalengine_api_docs_20261025/site contained 0.1.0a9. Its 28 package source/data files matched the inspected working tree, applying setup.py’s normal import rewrite for bundled root/vendor modules. Public checks ran from /tmp, using that installation target on PYTHONPATH, not checkout modules. The temporary target is a verification artifact and may disappear; users should follow the regular installation instructions above. No new wheel was built in this documentation pass.

# From the CausalEngine checkout, in the existing dependency environment:
PYTHONDONTWRITEBYTECODE=1 python -B -m unittest -q \
  test_scm test_graph_editor test_adjustment test_frontdoor \
  test_SBD test_identification test_indexed_identification
# 204 run; 202 passed; 2 skipped.

# From outside the checkout, with the matching package already installed:
PYTHONDONTWRITEBYTECODE=1 python -B -m unittest discover \
  -s /absolute/path/to/CausalEngine/tests -p 'test_public_api.py' -q
# 8 run; 8 passed.
PYTHONDONTWRITEBYTECODE=1 python -B -m unittest discover \
  -s /absolute/path/to/CausalEngine/tests -p 'test_cscm.py' -q
# 13 run; 13 passed.
PYTHONDONTWRITEBYTECODE=1 python -B /absolute/path/to/CausalEngine/tests/test_bdscm.py
# 12 checks; 12 passed.

For this run, the installed commands additionally used PYTHONPATH=/tmp/causalengine_api_docs_20261025/site and the interpreter above. The two checkout class-level skips request an optional flat-import UCA engine; the installed namespace contract was tested separately and passed. A skipped class does not mean all of its individual tests ran. See the Wiki validation table for each check’s scope. That historical checkout command used the file layout then available. Current source-only suites live in tests/checkout/; use the current Wiki commands rather than copying historical root-module names into a fresh clone.

The new mean example executed through public imports and returned (1000, 7) BD (1,). The CSCM query smoke confirmed UNKNOWN with default confounding and ACCEPT/BD with unconfounded=True. This pass also executed the guide’s existing Python examples, with the documented prerequisite imports. Generator-truth and finite-fixture checks are bounded regression evidence; they do not establish all-graph completeness or statistical coverage. The local graph-editor handler was exercised by tests; browser interaction/layout was not.

Candidate 0.1.0a10 verification

The candidate was built from tracked base 4caaa741 with only approved code, tests, assets, and documentation overlaid in /tmp. Existing unrelated dirty files were excluded. Wheel and source-distribution builds succeeded, including the HTML asset despite the checkout’s general HTML ignore rule. The wheel was installed offline with --no-deps into a temporary target using the existing Python 3.11.14 dependency runtime; the user’s environments were not upgraded.

• tests/test_weighting.py: 7 passed, including nonuniform source-metric optimality, normalization, signed weights, compatible/incompatible rank, actual finite-stage indicator balance, missing-target support failure, and actual affine-stage hard balance.
• tests/test_graph_tool.py: 4 passed, covering actual HTTP identification, negative/unfinished statuses, hidden/vector metadata, browser invocation mock, loopback request errors, and closeable server lifecycle.
• tests/test_public_api.py: 8 passed, including the additional public export and the existing scalar/vector identification/estimation contracts.

Headed browser QA confirmed node/edge editing, dragging, cycle rejection, vector/hidden roles, actual identification, and invalid-query errors. Desktop and 390-pixel mobile layouts passed; the final reload had no console errors or warnings. This QA covers the candidate HTML tool, separately from the historical graph-editor handler checks above.

The mathematical implementation review approved the Q-weighted objective, explicit normalization, finite support check, and preservation of the estimator selection. Exact fitted-design balance is not a guarantee of population balance or confidence-interval coverage. Fresh dependency resolution and other operating systems remain outside this validation.

Earlier checkout-organization record (October 26, 2026)

The text below retains the earlier organization record and its 212 executed cases. The source command has since been updated for the consolidated layout; it is not the command that produced that historical count:

PYTHONPATH=src python -B -m unittest discover -s tests/checkout -p 'test_*.py' -v

Run installed-package contracts separately, from outside the checkout, using the absolute test directory (for example python -B -m unittest discover -s /path/to/CausalEngine/tests -p test_public_api.py -v). Mixing flat-import source tests with the installed lazy-import contract in one process changes its premise.

The source-only SCM editor is grouped at tutorials/graph_editor.py beside its HTML asset. Launch python -B -m tutorials.graph_editor --port 8765 from the repository root; see the tutorial guide. Its experimental Korean UI generates models and sample rows. It is separate from the installed English identification tool. Retired estimation/simulation sources remain under archived/ as reference, preserving their pending compatibility edits. No estimator, weight solver, or installed tool behavior changes in this organization pass.

Organization validation passed all 212 executed test cases across the checkout, SCM, and installed public-API suites. Two optional flat-import UCA classes (22 methods) were not run; see the current validation record.

Candidate 0.1.0a11 verification (October 26, 2026)

A tracked 3ca36462 snapshot plus the approved 18-path overlay was built as a wheel and source distribution, then the wheel was installed offline with --no-deps in a temporary target using the existing Python 3.11.14 dependency runtime and XGBoost 2.0.3. Fresh dependency resolution and other OS/Python versions were not tested. Existing a9/a10 records above are historical.

• tests/test_uca_defaults.py: 5 run, 5 passed, 0 skipped. Actual XGBoost training occurred for default BD (2 calls), default FD (3), and explicit FD (3). The embedded front-door fixture completed NO_FIXED_PARTITION followed by MC ACCEPT and full-score execution with unlimited order/wall budgets. Its bounded order search can instead report UNKNOWN, which remains distinct from rejecting the criterion. Vector continuous covariates/outcomes, discrete action support, native/custom overrides and budget/scope guards were checked.
• tests/test_weighting.py: 7 run, 7 passed, 0 skipped; normalized minimum-L2 weights and finite/affine contracts remain intact.
• tests/test_public_api.py: 8 run, 8 passed, 0 skipped; old continuous-action pipeline checks now explicitly select the native backend.
• tests/checkout/test_identification.py’s CommonEngineTests: 19 run, 19 passed, 0 skipped, using this snapshot’s bundled engine explicitly. These optional tests were executed here, rather than inheriting earlier skips.
• tests/checkout/test_frontdoor.py’s UCAAdapterTests: 3 run, 3 passed, 0 skipped, using the bundled compiler for finite front-door expression checks.

The installed scalar/cluster demo retained native estimation and both cases returned ACCEPT. The new discrete-action default example also executed. Minimum-L2/fit_weights implementation bodies were unchanged, and the mathematical review approved the routing, learner/weight separation and point-score boundaries. These checks establish executable paths, not statistical accuracy or causal coverage.

UCA component performance pilot

The four-graph pilot compares Outcome/UCA plug-in, terminal IPW and the augmented UCA full score on paired data and exact binary-SCM truth. Its timing tables profile shared weight/regression/evaluation phases, rather than three independently executed estimators. Raw per-seed results and the reproduction command are linked from that report.

Graph-free mean estimators (introduced in a12)

Use estimate_bd, estimate_sbd, estimate_fd(covariates=...) or estimate_uca when the identifying functional is supplied by the user rather than found from a graph. The current a13 implementation uses the same XGBoost/minimum-L2 score engine, records user-assumed identification and does not synthesize graph verification. The earlier a11 pilot above remains a historical snapshot.

estimate_sbd declares ordered actions and new joint covariate blocks at each stage. It retains action-dependent time-varying covariate distributions and uses the same user-assumed point-score path, without graph verification.

Effects and whole-call pilot (a13)

estimate_ate and estimate_cate use paired regime scores. CATE regresses them on structurally supported baseline modifiers with a separate train/validation split; causal inference and conditional nuisance calibration remain assumed. The four-API pilot measures full public-call time, separately from the older shared-component timing.

Earlier README inference notes (a13; original experiment date not recorded)

The default backend='score', score_model=None returns a UCA point estimate and a conditional fitted-score summary, with no automatic causal confidence interval. It supports declared discrete actions and finite or continuous/vector covariates and outcomes where the compiler and weight design support the expression. Continuous fixed-point actions remain unsupported in this default: a Dirac target is not dominated by a diffuse source design. backend='native' explicitly retains the earlier plug-in estimators; the random-SCM demo above uses it. Explicit method='FD' also runs the default UCA/XGBoost score instead of a direct native front-door shortcut. XGBoost options use params and num_boost_round; an explicit regression_factory replaces the backward learner.

Opt-in backend='score', score_model='affine' executes the UCA full score under declared affine backward/feature closure, iid sampling, full-rank population designs, finite moments and nuisance-rate/stability assumptions. score_model='finite' instead requires fixed finite alphabets and positive population support. Intervals are coordinatewise and assumption-conditional, not data-certified causal coverage or simultaneous vector intervals; existing weak continuous-cluster FD cases achieved only 83–92% coverage.

Retained confounding-power demonstration

This is a generator diagnostic, not evidence that increasing power isolates one statistical difficulty.

Since 0.1.0a8, signed finite confounding_power=1.0 scales only the direct U term in the treatment logit. Its absolute value controls loading magnitude; its sign reverses that term’s direction. It leaves X, U, outcome mechanisms, Y0/Y1, and the ATE unchanged for the same task/data seeds. strength_range still samples the base loadings; task.loadings['A'] reports the actual scaled value.

Zero uses the same no-U-to-A path as unconfounded=True; the explicit flag overrides every power. task.confounding_power, task.unconfounded, and task.confounder_metadata report the effective setting. This is a structural assignment control, not a promise of monotonic observational bias. At zero, X-to-A and X-to-Y paths can still make an unadjusted observed difference biased.

The following deliberately specified Gaussian example has population ATE 0.25. Its base treatment is independent, and its outcome is Y = 0.25*A + loading_Y*U + 0.1*noise_Y. The power direction is fixed using mechanism coefficients before observing outcomes. There is no seed search. All rows and potential outcomes are shared across powers. It illustrates sign reversal; it is not a guarantee for every randomly generated outcome mechanism.

import numpy as np
from causalengine import CSCM, ConfounderOptions
from causalengine.bdscm import CovariateOptions, TreatmentOptions, OutcomeOptions, F_Y

class LinearOutcome:
    def __init__(self, **options):
        pass
    def mean(self, x, a, *, score_offset=0):
        return np.full(len(x), .25*a) + score_offset
    def __call__(self, x, a, noise, *, score_offset=0):
        return self.mean(x, a, score_offset=score_offset) + .1*noise

assert 'power_demo' not in F_Y
F_Y['power_demo'] = LinearOutcome
try:
    options = dict(d_X=5, task_seed=42,
        covariates=CovariateOptions(mechanism='mixed'),
        treatment=TreatmentOptions(mechanism='independent'),
        outcome=OutcomeOptions(family='power_demo'),
        confounder=ConfounderOptions(strength_range=(1., 1.)))
    base = CSCM(**options)
    direction = -np.sign(base.loadings['A']*base.loadings['Y'])
    print('power observed_difference oracle_ATE gap treated_fraction e01 e99 extreme_fraction')
    for magnitude in (0., .5, 1., 2.):
        power = float(direction*magnitude)
        observed, truth = CSCM(**options, confounding_power=power).sample(30_000, data_seed=7)
        a, y, e = observed['A'], observed['Y'], truth['e_xu']
        difference = y[a==1].mean()-y[a==0].mean()
        ate = np.mean(truth['mu1_xu']-truth['mu0_xu'])
        e01, e99 = np.quantile(e, [.01, .99])
        print(*(round(float(v), 4) for v in (power, difference, ate, difference-ate,
            a.mean(), e01, e99, np.mean((e<.01)|(e>.99)))))
finally:
    del F_Y['power_demo']

The observed difference compares treated/control sample means; oracle ATE uses generator potential-outcome means and is not learned from observational data. Propensity quantiles and the fraction outside [0.01,0.99] expose worsening overlap. Greater power can make causal estimation harder even though the target ATE is unchanged. Custom registry entries should use a fresh name; this example removes its temporary entry afterward.