CPT: Efficient Deep Neural Network Training via Cyclic Precision
CPT, 类似 CosineAnnealingWarmRestarts 的 Precision 循环机制
Empirical
Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients
Q-GaLore, 对 GaLore 进一步施加低精度量化
SWALP: Stochastic Weight Averaging in Low-Precision Training
SWALP, 通过 SWA 稳定低精度训练
Scaling Laws for Online Advertisement Retrieval
快手, 广告场景下的 Scaling Laws
EMR-MERGING: Tuning-Free High-Performance Model Merging
EMR-MERGING, 多任务权重合并技术
QARM: Quantitative Alignment Multi-Modal Recommendation at Kuaishou
QARM, 多模态推荐对齐与量化
Language Representations Can be What Recommenders Need: Findings and Potentials
Next-token embedding 之于协同过滤
Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation
ReaRec, 多步序列推荐推理
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
探究 LLM 如何记忆和提取知识的实验性文章
A Self-Attentive Model for Knowledge Tracing
SAKT, 自注意力知识追踪