Replication archive: A Multiverse of Good and Bad Controls: Candidate Causal Graphs for Interpreting Model Robustness Analysis

Code and (public) data to reproduce the tables and figures of the paper. Preprint: arXiv:2609.16618. Author: Shoki Okubo (Toyo University).

• analysis
• analysis/baselines
• analysis/output
• data
• figures
• pkg_logs
• regeneration
• sim
• sim/baselines
• sim/output

FILE_MANIFEST.txt lists every file in this snapshot except itself, the .git history and the release-check records (RELEASE_CHECK*), which are written after the manifest.

Computational commit checked against the manuscript: 9b1ce4defb25edefd30e2586c8f50a480aac9502 — see RELEASE_CHECK.md (with RELEASE_CHECK_run.log and RELEASE_CHECK_sessionInfo.txt when a clean-copy run was made). Later commits change documentation and the release record only — git diff --stat 9b1ce4defb25edefd30e2586c8f50a480aac9502 HEAD lists them — so the scripts, data and outputs are those of the checked commit; after any change to code or outputs the release check is rerun and this line is regenerated. Tag matching the posted preprint version: arxiv-2609.16618v1.

1. Install R (>= 4.1) and the packages pinned in analysis/README.md (the install command is given under "Quick start" below).
2. Run the scripts in each folder in the order given by their headers; outputs are written to output/. Where the folder contains its own analysis/README.md, that file is the authoritative description: execution order, which scripts generate results and which collect them, the environment of the shipped outputs and runtimes, and the map from every table, figure and script-produced number to a script and an output file. Where a stage_figures.sh is present, run it as bash stage_figures.sh figures from this folder (or bash ../stage_figures.sh <dest> from analysis/): it copies the manuscript figures from the outputs to figures/; the PDF build itself (Quarto) is not part of the numerical reproduction.
3. The applications use the public data files included in data/ (hurricanes.csv; nsw_dw.rda and cps_controls.rda; CPS1985.rda; Wages.rda); the simulations generate their own data. No licensed microdata are included.

Package: remotes::install_github("sokubo/[email protected]") — the version that produced the shipped outputs (test logs in pkg_logs/). The earlier version 0.1.2 (sokubo/[email protected]) is compatible: it reproduces the same values and differs only in reporting (printed labels and a message; see analysis/README.md). CRAN packages: MASS, ggplot2, glmnet (and parallel). Environment of the shipped outputs: R 4.6.0 on macOS 15.7.3 (arm64), dagmv 0.1.3, MASS 7.3-65, ggplot2 4.0.3, glmnet 5.0 (regeneration/REGEN_R460_sessionInfo.txt, RELEASE_CHECK_sessionInfo.txt). The complete default sequence (about 32 minutes on two cores — 1,899 s in the release check — of which phaseB_bootstrap.R takes 17):

cd sim      && Rscript poc_simulation.R && Rscript sim_main.R && Rscript sim_variants.R \
            && Rscript sim_random_graphs.R && Rscript sim_selectors.R && Rscript phaseB_mcse.R
cd ../analysis && Rscript reanalysis_hurricane.R && Rscript reanalysis_hurricane_full.R \
            && Rscript reanalysis_lalonde.R && Rscript reanalysis_union.R \
            && Rscript phaseB_bootstrap.R && Rscript phaseB_roles.R && Rscript phaseB_dag_figures.R \
            && Rscript phaseB_regression_checks.R && Rscript phaseB_simplex.R \
            && Rscript phaseB_nb_failure_pattern.R && Rscript phaseB_history_comparison.R \
            && Rscript phaseB_tables.R
cd .. && bash stage_figures.sh figures

sim/output/ and analysis/output/ are generated by this sequence (set them aside before a clean run); analysis/baselines/ and sim/baselines/ are archived summaries of earlier algorithms, read as inputs by phaseB_history_comparison.R and never regenerated — keep them in place. regeneration/ holds the record of the regeneration under R 4.6.0 and its comparison with the earlier R 4.3.3/Linux outputs (what that comparison covers, and the subsequent rerun of the analysis/ half after the message-language fix, are stated in analysis/README.md). RELEASE_CHECK.md records the anonymous download, manifest comparison and clean-copy re-run of the checked commit.

Okubo, S. (2026). A Multiverse of Good and Bad Controls: Candidate Causal Graphs for Interpreting Model Robustness Analysis. arXiv preprint arXiv:2609.16618.