1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
Adam 预训练的 1-bit SGD 优化方法
Error Compensation
MICROADAM: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence
MicroAdam, 通过梯度稀疏化以及 error compensation 实现轻量的优化器