Methodology & honest reporting

What was planned in advance, what was discovered afterwards, and how every number on this site was produced

1 · Why this page exists

Both artifacts on this site are exploratory self-directed projects built for PhD application evidence, not published papers. The purpose of this page is to separate what was committed to before seeing data from what emerged afterwards, to state the exact statistical caliber, and to list where every raw datum and script lives — so a reviewer can audit rather than take claims on faith.

2 · Exploratory phase vs confirmatory phase

Confirmatory (pre-registered)

Hypothesis written 2026-10-01 in PRE_REGISTRATION.md before any Hopper result was inspected.
Primary contrast: Baseline vs RewardScaled on asymptotic deterministic eval return.
Budget, seeds, eval protocol, and endpoint defined up front.
Result: null — p = 0.953; reported as-is.

Exploratory (post-hoc, uncorrected)

FixedAlpha control arm added after the first 60k pass to separate reward scaling from the auto-tuned α.
Pairwise Welch tests against FixedAlpha and the one-way ANOVA are descriptive only.
rl-deploy-bench pitfall catalog is engineering observation, not a controlled experiment.
No effect here is claimed as confirmed; all are labeled exploratory.

3 · Statistical caliber (Study 1)

Item	Caliber used
Unit of analysis	One per-seed tail mean per run; n = 3 seeds per arm (not individual eval episodes).
Asymptotic window	Last 4 deterministic eval points (105k–120k steps), mean per seed.
Dispersion	Sample SD, ddof = 1; t-based 95% CI with df = n−1 = 2 (t_0.975,2 = 4.303).
Pairwise test	Welch t-test (`scipy.stats.ttest_ind(equal_var=False)`); mean-difference CI on Welch-Satterthwaite df.
Omnibus	One-way ANOVA (`scipy.stats.f_oneway`), uncorrected.
Effect size	Cohen's d and Hedges' g for the primary contrast; reported descriptively only at n = 3.
Power	No power analysis was run. n = 3 with this seed variance has essentially no power; "p > 0.05" means not detectable, not "no effect".
Multiple comparisons	Post-hoc pairwise tests are not corrected (no Bonferroni/Holm); they are descriptive.

Recomputation: docs/assets/recompute_stats.py reads the committed raw CSVs directly and writes docs/assets/results.json; independent of plot.py. Numbers on the study page match both the README summary and this independent pass.

4 · Pre-registered vs post-hoc claims

Claim / analysis	Status	Evidence source
RewardScaled ≥ Baseline on asymptotic return	Pre-registered	`PRE_REGISTRATION.md`; tested Baseline vs RewardScaled, p = 0.953
Protocol: 120k steps, 3 seeds, 5 eval episodes/5k steps, last-4 window	Pre-registered	`PRE_REGISTRATION.md`, `configs/*.yaml`
Hypothesis #2: higher eval at 50k (faster early learning)	Pre-registered secondary, descriptive only	Single 50k eval point per seed; Baseline ≈ 332, RewardScaled ≈ 354, FixedAlpha ≈ 667; no inferential test; `results.json` `early_learning_50k_descriptive`
FixedAlpha arm (α = 0.2 frozen, scale 0.1)	Post-hoc control	Added after 60k pass; `configs/sac_hopper_fixedalpha.yaml`
Baseline vs FixedAlpha, FixedAlpha vs RewardScaled Welch tests	Post-hoc, uncorrected	`docs/assets/results.json`
One-way ANOVA F = 0.22, p = 0.811	Post-hoc, descriptive	`scipy.f_oneway` over the three tail-mean vectors
Apparent ~3× win at the 60k pass being overturned	Narrative, old CSVs not committed	`RESEARCH_RETRO.md`; not recomputed here
Training dynamics (final α, final Q₁ per seed)	Descriptive	Last train row of each committed CSV; `recompute_stats.py`
rl-deploy-bench pitfall taxonomy	Engineering observation	Committed fix commits, `CHANGELOG.md`, regression tests

5 · Data & code availability

Raw data (Study 1): per-run CSVs committed at logs/hopper_{baseline,rewardscale,fixedalpha}_s{0,1,2}/progress.csv; every train row (α, Q₁, return) and every eval point (deterministic return) is present.
Reproduction code (Study 1): train.py, src/ SAC implementation, plot.py; independent statistics docs/assets/recompute_stats.py; machine-readable results docs/assets/results.json.
Tests (Study 1): tests/test_core.py; CI learn gate ci_check.py; configs under configs/.
Study 2: full source under src/rl_deploy_bench/, regression suite tests/test_core.py, CHANGELOG.md (v1.1.0), API.md, CI matrix in .github/workflows/ci.yml.
No cherry-picking: all three seeds per arm are committed, including the low ones; no run was dropped for being inconvenient.

6 · Reproducibility limits, stated plainly

Single environment (Hopper-v4), 120k steps, CPU only; findings do not generalize to other envs or 1M-step budgets.
n = 3; confidence intervals are wide and overlap; the study can only detect very large effects.
Environment pin: the pre-registration named gymnasium 1.3.0 / mujoco 3.14.0; committed logs were produced on gymnasium 1.4.0 / mujoco 3.15.0. The discrepancy is logged in the README, not retro-edited into the pre-registration; Hopper-v4 reward semantics are unchanged across these minor bumps.
Multi-threaded CPU torch reductions are not bit-deterministic; re-runs reproduce the qualitative result, not identical floats, which is why CI uses a loose learn gate rather than exact match.
rl-deploy-bench was exercised on CPU; TensorRT and Jetson/edge paths are reviewed code, not measured on device. No speedup number on this site comes from a device benchmark that was never run.

7 · About the author

Daizhi Liao (GitHub: dafahaha) — Network Engineering, Guangzhou University; interests in reinforcement learning, off-policy stability, embodied AI, and empirical auditing of deployed systems.

embodied-offpolicy-study · rl-deploy-bench · TransitTruth (live demo) · personal site