Addressing Representation Collapse in Vector Quantized Models with One Linear Layer
SimVQ, 坐标变换替代可学习 Codebook
Empirical
Restructuring Vector Quantization with The Rotation Trick
一种利用 Rotation Trick 来替代 STE 的方案
Let’s Verify Step by Step
来自 OpenAI 的 process supervision
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
来自 DeepSeek 的 process supervision
All in One: Multi-Task Prompting for Graph Neural Networks
统一 graph/edge/node-level 的 graph prompt
Transformers need glasses! Information Over-Squashing in Language Tasks
LLM Representational Collapse
Data Augmentation as Free Lunch: Exploring the Test-Time Augmentation for Sequential Recommendation
TTA, Test-Time Augmentation
CPT: Efficient Deep Neural Network Training via Cyclic Precision
CPT, 类似 CosineAnnealingWarmRestarts 的 Precision 循环机制
Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients
Q-GaLore, 对 GaLore 进一步施加低精度量化
SWALP: Stochastic Weight Averaging in Low-Precision Training
SWALP, 通过 SWA 稳定低精度训练