May 7, 2025GaLore: Memory-Efficient LLM Training by Gradient Low-Rank ProjectionGaLore, 低秩空间中的梯度投影以及权重更新NoteLightweightLow-PrecisionOptimizerSVDTheoreticalICML2024
May 7, 2025Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank GradientsQ-GaLore, 对 GaLore 进一步施加低精度量化NoteLightweightLow-PrecisionOptimizerSVDEmpiricalArXiv2024