EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register Tokens
通过减小 KV cache size 加速 LLMRec 的推理
LLM
Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation
LC-Rec, LLM + RQ-VAE + 丰富的多任务训练
Base of RoPE Bounds Context Length
讨论 RoPE base 对于相似 Tokens 感知能力的影响
Round and Round We Go! What makes Rotary Positional Encodings useful?
理解 RoPE 的高低频
Transformers need glasses! Information Over-Squashing in Language Tasks
LLM Representational Collapse
Language Representations Can be What Recommenders Need: Findings and Potentials
Next-token embedding 之于协同过滤
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
探究 LLM 如何记忆和提取知识的实验性文章