Dynamic Chunking for End-to-End Hierarchical Sequence Modeling
符号序列的自动切分, 探究非 Subword Tokenizer 的可能性
Blogs
SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer
TiTok 框架 + Softmax 版 Vector Quantization 以期更高的压缩比
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
多尺度 Transformer, 探究非 Subword Tokenizer 的可能性
SpaceByte: Towards Deleting Tokenization from Large Language Modeling
探究非 Subword Tokenizer 的可能性
Temporal Cross-Effects in Knowledge Tracing
借鉴 Hawkes Process 思想建模知识掌握的交叉影响和衰减
EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register Tokens
通过减小 KV cache size 加速 LLMRec 的推理
Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations
COBRA, 链接离散编码和稠密表示的尝试
Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation
LC-Rec, LLM + RQ-VAE + 丰富的多任务训练
Sinkhorn Distance and Sinkhorn-Knopp Algorithm
关于利用 Sinkhorn 距离求解离散最优传输问题的记录
OneRec: Unifying Retrieve and Rank with Generative Recommender and Preference Alignment
OneRec, 端到端的推荐模型