Junwei Quan

MScAC in Applied Computing (AI)
University of Toronto

I work on reinforcement learning and world models for robotics. I am interested in agents that stay reliable in difficult situations: over long horizons, off the training distribution, and in the moments when the right move is to plan at test time rather than react. I also work on AI alignment and safety for large language models, from post-training to deployed agents.

I am currently a master's student at the University of Toronto, supported by the Ontario Vector Institute AI Master's Entrance Scholarship. Happy to chat about related topics!

Portrait of Junwei Quan

Research

AI systems can look capable on the problems they were trained and evaluated on, then become surprisingly brittle when conditions change or decisions stretch over time. I want to understand where that reliability breaks down and how an agent can recognize, reason through, and recover from those failures at test time. Four threads:

Long-horizon control

AI systems are often good at making the next decision, but much less reliable when success depends on hundreds of decisions working together. Small mistakes accumulate, early assumptions become outdated, and a system can drift far from its original goal without realizing it.

Out-of-distribution behaviour

The hardest situations are often the ones an AI system has never seen before. A model may perform extremely well when new inputs resemble its training data, yet become unreliable when it encounters unfamiliar combinations, environments, or strategies.

Test-time planning

Much of what a model learns is richer than what appears in its immediate output. Its representations may already contain useful information about what is possible, what is nearby, or which paths are promising—but a single forward pass gives the model little opportunity to explore that structure.

Alignment and safety

A capable model is not necessarily one that does what we intend. Its behaviour depends on choices made during post-training, and once it acts as an agent, it may game its objective, follow harmful instructions, or take unsafe actions, often in ways that are hard to notice from the outside. Understanding how training shapes this behaviour, and detecting when it goes wrong, matters more as models gain autonomy.

News

Publications

* denotes equal contribution.

01 Reinforcement learning & world models

Test-time planning and policy selection for goal-conditioned agents.

  1. World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning

    J. Quan, E. Opryshko, N. Rhinehart, I. Gilitschenski

    arXiv preprint

    No single offline goal-conditioned RL algorithm wins everywhere, so WMPA treats a bank of frozen policies as a portfolio: it rolls each one out in a learned world model, scores the imagined futures with a shared goal-conditioned value function, and runs the best policy for a short commitment interval before choosing again — no retraining required. On 18 state-based OGBench datasets, macro-average success rises from 44% (best single policy per dataset) to 58%, with +33 points on cube-double-play and +36 on scene-play.

    arXiv project page code

  2. Test-Time Graph Search for Goal-Conditioned Reinforcement Learning

    E. Opryshko*, J. Quan*, C. Voelcker, Y. Du, I. Gilitschenski

    ICML 2026

    Samples states from an offline dataset into a weighted graph, reinterprets a pre-trained goal-conditioned value function as a distance signal, and runs Dijkstra search to emit executable subgoal sequences — letting a fixed policy decompose and plan long-horizon tasks with no additional training. On OGBench pointmaze-giant-stitch-v0, HIQL goes from 0.0% to 80.9% and GCIQL from 0.0% to 98.0%.

    arXiv project page code

02 Language model training & safety

Training language models, from pretraining to RL post-training, and catching unsafe behaviour once they are deployed.

  1. RH-Detect: A Unified Benchmark for Reward Hacking Detection

    J. Quan*, E. Opryshko*, R. Subramani, I. Gilitschenski

    arXiv preprint

    Brings reward-hacking-relevant subsets of eleven public datasets — 92,761 rows across six behaviour categories — into one schema, and evaluates off-the-shelf language models as detectors with no additional training. The best reaches a pooled AUROC of 0.962 and accuracy above 93%, but the four strongest models are 10.7–15.9 points less accurate on multi-turn tool-use trajectories than elsewhere — a key gap for deployment-time monitoring. Leave-one-source-out SFT on the benchmark improves AUROC on five of six held-out sources.

    arXiv

  2. MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Scaling of Diffusion Language Models

    CH. Chao, WF. Sun, J. Quan, CY. Lee, R. G. Krishnan

    EMNLP 2026main conference

    We show that how tokens are represented matters for scaling diffusion language models. MDM-Prime-v2 uses binary sub-token representations and index shuffling to make partial masking more effective, leading to stronger scaling and the best average zero-shot performance across eight reasoning benchmarks among comparable 1.1B-scale autoregressive and diffusion models.

    arXiv project page code

Also: an ICSE 2025 demo paper and an MO2RE 2025 paper, both in software engineering.

Education & Awards

2026.09 — 2027.12

MScAC in Applied Computing — Artificial Intelligence

University of Toronto

2022.09 — 2026.06

BSc, Computer Science Specialist (AI focus) & Data Science Specialist

University of Toronto — Mathematics minor · CGPA 3.91 / 4.00