arxivcs.LGmath.OCstat.ML2026-07-03
On the Convergence of Adam, Revisited
Steven Heilman, Sampad Mohanty
We show that projected Adam for online optimization with arbitrary moment decay parameters $β_1,β_2\in[0,1)$ can have average regret bounded away from zero. A similar result of Reddi-Kale-Kumar from 2018 required $β_1<\sqrt{β_2}$. Similar to their result, we use a three-periodic…