Yan DAI (戴言)
Operations Research PhD Student at 
Previously CS Bachelor's Student at 
About
Yan Dai's Adam is FTRL identifies the Adam optimizer as an implementation of the Follow-the-Regularized-Leader framework. Dai is a PhD student at the MIT Operations Research Center, where they are co-advised by Patrick Jaillet and Negin Golrezaei. Their research investigates strategic behaviors in economic systems using online learning, mechanism design, and RL theory. Dai authored Normalization in SAM to analyze the geometric effects of weight normalization and developed the Banker Online Mirror Descent framework for bandit learning with delays. They received a Best Paper Award at SIGMETRICS 2025 for Adversarial Network Optimization under Bandit Feedback. Dai earned a bachelor’s degree from the Tsinghua University Yao Class and has held research positions at Microsoft Research and the University of Washington.
Experience
Operations Research PhD Student
2024 – Present
Co-advised by Patrick Jaillet and Negin Golrezaei.
2024 – Present
Research Assistant, Operations Research
Sep 2024 – Present
Worked with Patrick Jaillet and Negin Golrezaei.
Sep 2024 – Present
Research Intern
May 2026 – Present
Microsoft Research Economics and Computer Science Group · Cambridge, MA
Mentored by Alex Slivkins and Brendan Lucier; worked closely with Nicole Immorlica.
May 2026 – Present
Tsinghua University · Beijing, China
Bachelor's Thesis Student
Feb 2024 – Jun 2024
Advised by Longbo Huang.
Feb 2024 – Jun 2024
CS Bachelor's Student
2020 – 2024
2020 – 2024
Research Assistant
Jun 2021 – Dec 2021
Worked with Longbo Huang.
Jun 2021 – Dec 2021
Visiting Student, CSE
Jun 2023 – Aug 2023
University of Washington · Seattle, WA
Hosted by Simon S. Du.
Jun 2023 – Aug 2023
Visiting Student, LID
Feb 2023 – May 2023
Massachusetts Institute of Technology · Cambridge, MA
Hosted by Suvrit Sra.
Feb 2023 – May 2023
Visiting Student, CS
Feb 2022 – Dec 2022
University of Southern California · Remote
Worked with Haipeng Luo.
Feb 2022 – Dec 2022
Papers14
uniINF: Best-of-Both-Worlds Algorithm for Parameter-Free Heavy-Tailed MABs
In this paper, we present a novel algorithm, uniINF, for the Heavy-Tailed Multi-Armed Bandits (HTMAB) problem, demonstrating robustness and adaptability in both stochastic and adversarial environments. Unlike the stochastic MAB setting where loss distributions are stationary with time, our study extends to the adversarial setup, where losses are generated from heavy-tailed distributions that depend on both arms and time. Our novel algorithm `uniINF` enjoys the so-called Best-of-Both-Worlds (BoBW) property, performing optimally in both stochastic and adversarial environments without knowing the exact environment type. Moreover, our algorithm also possesses a Parameter-Free feature, i.e., it operates without the need of knowing the heavy-tail parameters a-priori. To be precise, uniINF ensures nearly-optimal regret in both stochastic and adversarial environments, matching the corresponding lower bounds when is known (up to logarithmic factors). To our knowledge, uniINF is the first parameter-free algorithm to achieve the BoBW property for the heavy-tailed MAB problem. Technically, we develop innovative techniques to achieve BoBW guarantees for Parameter-Free HTMABs, including a refined analysis for the dynamics of log-barrier, an auto-balancing learning rate scheduling scheme, an adaptive skipping-clipping loss tuning technique, and a stopping-time analysis for logarithmic regret.
Adversarial Network Optimization under Bandit Feedback: Maximizing Utility in Non-Stationary Multi-Hop Networks
Stochastic Network Optimization (SNO) concerns scheduling in stochastic queueing systems. It has been widely studied in network theory. Classical SNO algorithms require network conditions to be stationary with time, which fails to capture the non-stationary components in many real-world scenarios. Many existing algorithms also assume knowledge of network conditions before decision, which rules out applications where unpredictability presents. Motivated by these issues, we consider Adversarial Network Optimization (ANO) under bandit feedback. Specifically, we consider the task of *i)* maximizing some unknown and time-varying utility function associated to scheduler's actions, where *ii)* the underlying network is a non-stationary multi-hop one whose conditions change arbitrarily with time, and *iii)* only bandit feedback (effect of actually deployed actions) is revealed after decisions. Our proposed `UMO2` algorithm ensures network stability and also matches the utility maximization performance of any "mildly varying" reference policy up to a polynomially decaying gap. To our knowledge, no previous ANO algorithm handled multi-hop networks or achieved utility guarantees under bandit feedback, whereas ours can do both. Technically, our method builds upon a novel integration of online learning into Lyapunov analyses: To handle complex inter-dependencies among queues in multi-hop networks, we propose meticulous techniques to balance online learning and Lyapunov arguments. To tackle the learning obstacles due to potentially unbounded queue sizes, we design a new online linear optimization algorithm that automatically adapts to loss magnitudes. To maximize utility, we propose a bandit convex optimization algorithm with novel queue-dependent learning rate scheduling that suites drastically varying queue lengths. Our new insights in online learning can be of independent interest.
Follow-the-Perturbed-Leader for Adversarial Markov Decision Processes with Bandit Feedback
We consider regret minimization for Adversarial Markov Decision Processes (AMDPs), where the loss functions are changing over time and adversarially chosen, and the learner only observes the losses for the visited state-action pairs (i.e., bandit feedback). While there has been a surge of studies on this problem using Online-Mirror-Descent (OMD) methods, very little is known about the Follow-the-Perturbed-Leader (FTPL) methods, which are usually computationally more efficient and also easier to implement since it only requires solving an offline planning problem. Motivated by this, we take a closer look at FTPL for learning AMDPs, starting from the standard episodic finite-horizon setting. We find some unique and intriguing difficulties in the analysis and propose a workaround to eventually show that FTPL is also able to achieve near-optimal regret bounds in this case. More importantly, we then find two significant applications: First, the analysis of FTPL turns out to be readily generalizable to delayed bandit feedback with order-optimal regret, while OMD methods exhibit extra difficulties (Jin et al., 2022). Second, using FTPL, we also develop the first no-regret algorithm for learning communicating AMDPs in the infinite-horizon setting with bandit feedback and stochastic transitions. Our algorithm is efficient assuming access to an offline planning oracle, while even for the easier full-information setting, the only existing algorithm (Chandrasekaran and Tewari, 2021) is computationally inefficient.
Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
Despite the success of the Adam optimizer in practice, the theoretical understanding of its algorithmic components still remains limited. In particular, most existing analyses of Adam show the convergence rate that can be simply achieved by non-adative algorithms like SGD. In this work, we provide a different perspective based on online learning that underscores the importance of Adam's algorithmic components. Inspired by Cutkosky et al. (2023), we consider the framework called online learning of updates/increments, where we choose the updates/increments of an optimizer based on an online learner. With this framework, the design of a good optimizer is reduced to the design of a good online learner. Our main observation is that Adam corresponds to a principled online learning framework called Follow-the-Regularized-Leader (FTRL). Building on this observation, we study the benefits of its algorithmic components from the online learning perspective.
Non-Monetary Mechanism Design without Distributional Information: Using Scarce Audits Wisely
We study a repeated resource allocation problem with strategic agents where monetary transfers are disallowed and the central planner has no prior information on agents' utility distributions. In light of Arrow's impossibility theorem, acquiring information about agent preferences through some form of feedback is necessary. We assume that the central planner can request powerful but expensive audits on the winner in any round, revealing the true utility of the winner in that round. We design a mechanism achieving -independent social welfare regret while only requesting audits in expectation, where is the number of agents and is the number of rounds. We also show an lower bound on the regret and an lower bound on the number of audits when having low regret. Algorithmically, we show that incentive-compatibility can be mostly enforced via the imposition of adaptive future punishments, where the audit probability is inversely proportional to the winner's future winning probability. To accurately estimate such probabilities in presence of strategic agents, who may adversely react to any potential misestimate, we introduce a flagging component that allows agents to flag any biased estimate (we show that doing so aligns with individual incentives). On the technical side, without a unique and known distribution, one cannot apply the revelation principle and conclude that truthful reporting is exactly an equilibrium. Instead, we characterize the equilibrium via a reduction to a simpler auxiliary game, in which agents cannot strategize until close to the end of the game; we show equilibria in this game can induce equilibria in the actual, fully strategic game. The tools developed therein may be of independent interest for other mechanism design problems in which the revelation principle cannot be readily applied.
Variance-Aware Sparse Linear Bandits
It is well-known that for sparse linear bandits, when ignoring the dependency on sparsity which is much smaller than the ambient dimension, the worst-case minimax regret is where is the ambient dimension and is the number of rounds. On the other hand, in the benign setting where there is no noise and the action set is the unit sphere, one can use divide-and-conquer to achieve regret, which is (nearly) independent of and . In this paper, we present the first variance-aware regret guarantee for sparse linear bandits: , where is the variance of the noise at the -th round. This bound naturally interpolates the regret bounds for the worst-case constant-variance regime (i.e., ) and the benign deterministic regimes (i.e., ). To achieve this variance-aware regret guarantee, we develop a general framework that converts any variance-aware linear bandit algorithm to a variance-aware algorithm for sparse linear bandits in a "black-box" manner. Specifically, we take two recent algorithms as black boxes to illustrate that the claimed bounds indeed hold, where the first algorithm can handle unknown-variance cases and the second one is more efficient.
Policy Regret for Embedding Model Routing: Contextual Bandits with Low-Rank Experts
Modern recommendation systems increasingly rely on dynamically routing diverse queries to multiple embedding models. Despite its practical significance, this problem remains poorly understood under realistic conditions like adversarial queries, bandit feedback, and limited observability of models. We formalize embedding model routing as an adversarial contextual linear bandit with low-rank experts, where contexts are queries, actions are items, and experts are the embedding models working on low-rank latent representation spaces. We first establish that standard regret notions suffer from structural misspecification or statistical intractability, and we identify a log-quadratic policy class that is expressive enough to capture query-dependent model routing, yet structured enough to allow efficient online learning. Focusing on this log-quadratic policy optimization problem under bandit feedback -- which is of independent interest -- we propose a policy gradient algorithm called Hypentropy Policy Gradient (HPG). It provably adapts to the unknown low-rank structure under incomplete information and attains linearized policy regret -- where , and are the intrinsic rank of the experts, the number of models, and the number of rounds -- thus avoiding a curse of dimensionality. We provide computationally efficient and parameter-free implementation of HPG.
The Crucial Role of Normalization in Sharpness-Aware Minimization
Sharpness-Aware Minimization (SAM) is a recently proposed gradient-based optimizer (Foret et al., ICLR 2021) that greatly improves the prediction performance of deep neural networks. Consequently, there has been a surge of interest in explaining its empirical success. We focus, in particular, on understanding the role played by normalization, a key component of the SAM updates. We theoretically and empirically study the effect of normalization in SAM for both convex and non-convex functions, revealing two key roles played by normalization: i) it helps in stabilizing the algorithm; and ii) it enables the algorithm to drift along a continuum (manifold) of minima -- a property identified by recent theoretical works that is the key to better performance. We further argue that these two properties of normalization make SAM robust against the choice of hyper-parameters, supporting the practicality of SAM. Our conclusions are backed by various experiments.
Refined Sample Complexity for Markov Games with Independent Linear Function Approximation
Markov Games (MG) is an important model for Multi-Agent Reinforcement Learning (MARL). It was long believed that the "curse of multi-agents" (i.e., the algorithmic performance drops exponentially with the number of agents) is unavoidable until several recent works (Daskalakis et al., 2023; Cui et al., 2023; Wang et al., 2023). While these works resolved the curse of multi-agents, when the state spaces are prohibitively large and (linear) function approximations are deployed, they either had a slower convergence rate of or brought a polynomial dependency on the number of actions -- which is avoidable in single-agent cases even when the loss functions can arbitrarily vary with time. This paper first refines the AVLPR framework by Wang et al. (2023), with an insight of designing *data-dependent* (i.e., stochastic) pessimistic estimation of the sub-optimality gap, allowing a broader choice of plug-in algorithms. When specialized to MGs with independent linear function approximations, we propose novel *action-dependent bonuses* to cover occasionally extreme estimation errors. With the help of state-of-the-art techniques from the single-agent RL literature, we give the first algorithm that tackles the curse of multi-agents, attains the optimal convergence rate, and avoids dependency simultaneously.
Metrics
Citations per year
Coauthors
