BEiT: BERT Pre-Training of Image Transformers
Visual Tokens & Masked Image Modeling
ICLR
Restructuring Vector Quantization with The Rotation Trick
一种利用 Rotation Trick 来替代 STE 的方案
Let’s Verify Step by Step
来自 OpenAI 的 process supervision
Round and Round We Go! What makes Rotary Positional Encodings useful?
理解 RoPE 的高低频
CPT: Efficient Deep Neural Network Training via Cyclic Precision
CPT, 类似 CosineAnnealingWarmRestarts 的 Precision 循环机制
Language Representations Can be What Recommenders Need: Findings and Potentials
Next-token embedding 之于协同过滤