DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
GRPO: Group Relative Policy Optimization
Seminal
Proximal Policy Optimization Algorithms
PPO
REINFORCE Algorithm
著名的 REINFORCE 算法
Trust Region Policy Optimization
PPO 的前身
On the Reliability of Sampling Strategies in Offline Recommender Evaluation
不同采样策略在不同曝光偏差下的区分性, 鲁棒性, 一致性
Product Quantization for Nearest Neighbor Search
将 Product Quantization 用于搜索以降低存储占用
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
多尺度 Transformer, 探究非 Subword Tokenizer 的可能性
Restructuring Vector Quantization with The Rotation Trick
一种利用 Rotation Trick 来替代 STE 的方案
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
探究 LLM 如何记忆和提取知识的实验性文章
Dynamic Key-Value Memory Networks for Knowledge Tracing
DKVMN, 带记忆结构的知识追踪