报告题目：Gradient-Tracked and Variance-Reduced Stochastic Quasi-Newton Methods for Decentralized Learning

报告人：凌青教授（中山大学）

报告时间： 2023年3月22日上午10:20-11:20

报告地点：龙洞校区行政楼610

主持人：刘玉兰

报告摘要：In this work, we investigate stochastic quasi-Newton methods for minimizing a finite sum of cost functions over a decentralized network. We develop a general algorithmic framework that incorporates stochastic quasi-Newton approximation with variance reduction so as to achieve fast convergence. At each time each node constructs a local, inexact quasi-Newton direction that asymptotically approaches the global, exact one. To be specific, (i) A local gradient approximation is constructed by using dynamic average consensus to track the average of variance-reduced local stochastic gradients over the entire network; (ii) A local Hessian inverse approximation is assumed to be positive definite with bounded eigenvalues, and we specify two fully decentralized stochastic quasi-Newton methods, damped regularized limited-memory DFP (Davidon-Fletcher-Powell) and damped limited-memory BFGS (Broyden-Fletcher-Goldfarb-Shanno), to locally construct such a Hessian inverse approximation without extra sampling or communication. Compared to the existing decentralized stochastic first-order methods, the proposed general framework introduces the second-order curvature information without incurring extra sampling or communication. With a fixed step size, we establish the conditions under which the proposed general framework linearly converges to an exact optimal solution.

专家简介：凌青，2001年与2006年于中国科学技术大学自动化系分别获得学士与博士学位，2006年至2009年担任密歇根理工大学电子工程与计算机科学系博士后研究员。2009年至2017年任教于中国科学技术大学自动化系，其间曾担任宾夕法尼亚大学电子与系统工程系访问学者、微软亚洲研究院铸星计划访问学者。2017年起担任中山大学计算机学院教授、博士生导师。在IEEE Transactions on Signal Processing、IEEE Transactions on Parallel and Distributed Systems、SIAM Journal on Optimization等杂志发表论文70余篇。获得IEEE信号处理协会青年作者最佳论文奖。担任IEEE Signal Processing Letters杂志资深编辑、IEEE Transactions on Network and Service Management杂志编委，曾担任IEEE Signal Processing Letters杂志编委。