Transformers need glasses! Information Over-Squashing in Language Tasks
LLM Representational Collapse
NeurIPS
MICROADAM: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence
MicroAdam, 通过梯度稀疏化以及 error compensation 实现轻量的优化器
EMR-MERGING: Tuning-Free High-Performance Model Merging
EMR-MERGING, 多任务权重合并技术
Deep Knowledge Tracing
DKT, 知识追踪
Recommender Systems with Generative Retrieval
TIGER, 向量量化生成式检索
Neural Discrete Representation Learning
VQ-VAE, 向量量化的开山之作