Fourth-year undergraduate (senior), Guangzhou University · Network Engineering
Applying to PhD programs in Fall 2027 · Embodied AI / Reinforcement Learning / Robotics
I lead a small set of my own tools and studies — things that test what a deployed model actually does, not what it claims to do. My open-source work began with a broad exploratory phase of small PRs across many RL/ML repos; I've since narrowed to a few in-depth upstream fixes, which is what the section below leads with.
An open-source system that checks whether an AI API relay actually serves the model it advertises.
Instead of trusting the relay's label, it probes the model's statistical behavior — random-number,
token, and latency distributions — and compares them against a baseline using chi-square / KS tests
with Bayesian updating. A guided offline demo walks through the flow; run the real audit locally
with transit-truth serve.
The accompanying extended technical report validates same-family discrimination (gpt-4o vs gpt-4o-mini,
TVD up to 0.90, 91.3% in-distribution resubstitution posterior — self-consistency on the baseline's
own reference, not held-out accuracy), focusing on fine-grained distinctions that
single-token fingerprinting does not reach.
A preregistered, single-machine CPU audit suite that disentangles a world model’s perceptual fidelity (next-frame MSE / SSIM / NLL, plus fixed-filter stand-ins for LPIPS / rollout FVD — deterministic random-conv filter banks, not learned VGG/AlexNet backbones) from its control utility (whether a planner can actually solve the task using it). A reduced pilot on one env (pixel_nav, 3 seeds × 3 capacities, 12k steps) found: H1 threshold met — pixel fidelity weakly tracks closed-loop return at |ρ|=0.32<0.4, but this is a weak, tentative exploratory signal only (n=9 pilot; the 95% CI crosses zero and Holm-adj p=1.0, so it is not statistically significant and not an established finding); control-relevant diagnostics did not (H2 not supported, ρ=0.26<0.6); INT8 compression did not degrade calibration faster than raw pixel fidelity (H3 not supported, n_pairs=9). This is a CPU reduced pilot, not the locked 200k-step × 5-seed × 2-env main run.
A Soft Actor-Critic implementation written from scratch in PyTorch (no Stable-Baselines3), run as a pre-registered two-arm controlled study with a logged third-arm (FixedAlpha) amendment on Hopper-v4 (CPU, 120k env steps, 3 seeds/arm): Baseline (reward scale 1.0) vs RewardScaled (0.1) vs FixedAlpha (scale=0.1, α fixed at 0.2; automatic temperature off). Mean return ± SD (final window 105k–120k, n=3/arm): Baseline 1085±698, RewardScaled 1118±623, FixedAlpha 1404±632. Welch p: Baseline–FixedAlpha 0.59, Baseline–RewardScaled 0.95, FixedAlpha–RewardScaled 0.61; ANOVA F=0.22, p=0.81 — no statistically distinguishable effect detected (n=3; t-intervals overlap heavily). The study's value: at 60k the reward-scaled arm looked ~3× better, but extending to the pre-registered 120k budget and adding the fixed-temperature control arm overturned that apparent gain as early seed noise — an honest pre-registered null. Raw CSVs are committed and curves reproduced by script.
A config-driven CLI that exports Stable-Baselines3 policies to ONNX/TorchScript and quantizes to FP16/INT8; benchmarks latency, throughput, and accuracy on x86 GPU and CPU via ONNX Runtime, with auto-generated HTML reports. An experimental standalone TensorRT engine-builder is included but not yet wired into the benchmark harness — a reproducible harness for the accuracy–latency tradeoff of quantized RL policies. Jetson/edge is a target deployment path, not yet measured on device.
After an early, broad exploratory phase of small docs/fix PRs across many RL/ML repos, I've narrowed to a few in-depth upstream fixes. Eight contributions are merged: three code fixes below, and five community/documentation entries listed separately (not code fixes).
Merged code fixes
| Project | PR | What I fixed |
|---|---|---|
| microsoft/TextWorld | #377 merged | Fixed env.step crash on multi-command input (non-greedy score parsing + regression test). |
| shmuma/ptan | #57 merged | Fixed Gymnasium 5-tuple API compatibility in experience sources. |
| learnsyslab/gym-pybullet-drones | #319 merged | Converted print()+exit() error paths to raised exceptions across 8 files. |
Merged community & documentation
bloat16 → bf16 in the BFloat16 support check.Selected code PRs currently under review (not yet merged) in upstream RL/ML infrastructure:
| Project | PR | What I fixed |
|---|---|---|
| thu-ml/tianshou | #1315 under review | Fixed pickled policy restore under PyTorch 2.6 weights_only default. |
| ContinualAI/avalanche | #1701 under review | Fixed MASPlugin raising on zero loss (falsy check). |
| uoe-agents/epymarl | #92 under review | Fixed registration of multi-digit versioned PettingZoo envs. |
| vllm-project/vllm | #57400 under review | Fixed system_fingerprint=null emitted under fingerprint-mode=none. |
| huggingface/datasets | #8633 under review | Accepted pathlib.Path in load/save_to_disk. |
| huggingface/diffusers | #14796 under review | Fixed undefined attribute in TangentialClassifierFreeGuidance. |
| google/brax | #676 under review | Fixed EpisodeWrapper metrics accumulation with action_repeat. |
| AI4Finance-Foundation/ElegantRL | #488 under review | Fixed argmax→max in get_cumulative_rewards. |
| Farama-Foundation/PettingZoo | #1465 under review | Fixed agent selector reinit in tictactoe reset. |
| MichalBortkiewicz/JaxGCRL | #65 under review | Fixed XML path resolution in arm manipulation envs. |
| pandas-dev/pandas | #68968 under review | Fixed to_html href escaping with render_links + escape. |
| RLE-Bench/RLE-Bench | #9 under review | Recorded per-trial status/reason/timing in summary.json; resolved embedded final-step reward from the step directory. |
Other early exploratory PRs across smaller RL/ML projects remain open or unanswered; they are part of the broad phase noted above, not deep work. Root-cause writeups live on my blog: the TextWorld multi-command regex fix, and a postmortem on numpy/scipy missing from requirements.
Agent skills & tools
Small, focused, offline tools I built to keep RL reporting and API-relay setup honest. They are zero-dependency local utilities (not services); each ships with its own test suite.
Research notes and figures from the offline-policy study are published as a GitHub Pages notebook.