论文
随笔
Tags
Slides
⤴
Search
## Markovian Pre-trained Transformer for Next-Item Recommendation
<h2 id="background-pre-training--fine-tuning">Background: Pre-training & Fine-tuning</h2> <ul> <li> <p><strong>Pre-training & Fine-tuning:</strong> 多个领域的致胜法宝</p> <ul> <li>(CV) ResNet, SimCLR, MAE, ViT …</li> <li>(NLP) BERT, GPT …</li> <li>(CV & NLP) CLIP, BLIP, SigLIP …</li> </ul> </li> <li> <p>Pre-training & Fine-tuning for Next-Item Recommendation:</p> <ul> <li>(美好愿景) 😄 quick deployment; 😄 better generalizability</li> <li>(研究现状) 😞 效果远远逊色于 domain-specific 模型</li> </ul> </li> </ul> <div class="slide-highlight"> 🤔什么阻碍了"推荐知识"的迁移? </div>
<h2 id="background-pre-training--fine-tuning-1">Background: Pre-training & Fine-tuning</h2> <ul> <li> <p><strong>Next-Item Recommendation:</strong></p> $$ [v_1, v_2, \ldots, v_t] \rightarrow v_{t+1} $$</li> <li> <p>推荐数据的异构性 (heterogeneity):</p> <ol> <li>diverse user behaviors</li> <li>non-negligible domain gaps</li> </ol> </li> </ul> <div class="slide-highlight"> 🤔什么样的"推荐知识"是可迁移的? </div>
<h2 id="rqi-transferable-capabilities">RQI: Transferable Capabilities</h2> <div class="slide-highlight"> 学好 XXX 有什么用, 生活里又用不到! </div> <ul> <li> <p><strong>Computer Vision:</strong></p> <ol> <li>(图像分类) 特征提取、模式识别 …</li> <li>(图像生成) 数据分布建模 …</li> </ol> </li> <li> <p><strong>Natural Language Processing:</strong></p> <ol> <li>(传统语料) 语义理解、语法保持</li> <li>(数学/代码) 逻辑推理</li> </ol> </li> </ul> <div class="slide-highlight"> 可迁移的是能力! </div>
<h2 id="transferable-capabilities-for-recommendation">Transferable Capabilities for Recommendation</h2> <ul> <li>推荐需要何种能力? 长短兴趣建模?</li> </ul> <div class="slide-img"> <img src="https://raw.githubusercontent.com/MTandHJ/blog_source/master/images/20260112202832.png" alt="Image" style="max-width: 100%; height: auto;margin: 0 auto;"> </div> <div class="slide-ref"> <div style="width: 100px; height: 1px; background: black; margin-bottom: 5px;"></div> <p style="margin: 2px 0;"> GRU4Rec: RNN; SASRec: Transformer; HSTU: Attention + Time-based positional encoding </p> </div>
<h2 id="transferable-capabilities-for-recommendation-1">Transferable Capabilities for Recommendation</h2> $$ \begin{array}{rl} \textcircled{\small 1} \text{ Chronologically ordered:} & \mathbb{P}\left(v_{t+1}\,|\,v_t, v_{t-1}, \ldots, v_1; \Theta \right), \\ \textcircled{\small 2} \text{ Partially shuffled:} & \mathbb{P}\left(v_{t+1}\,|\, v_t, \{v_1, v_2, \ldots \}; \Theta \right), \\ \textcircled{\small 3} \text{ Completely shuffled:} & \mathbb{P}\left(v_{t+1}\,|\,\{v_1, v_2, \ldots, v_t\}; \Theta \right) \end{array} $$<ul> <li> <p>$\textcircled{\small 1} \approx \textcircled{\small 2}$: 先进的序列推荐模型并没有依赖序列性做出更加复杂的推理 (即使 HSTU 引入了 Timestamps 信息)</p> </li> <li> <p>$\textcircled{\small 1}/\textcircled{\small 2} \gtrapprox \textcircled{\small 3}$: Latest interaction 至关重要</p> </li> <li> <p>上述结论与数据集预处理方式、优化目标、模型表达能力无关</p> </li> </ul>
<h2 id="markovian-nature-of-next-item-prediction">Markovian Nature of Next-Item Prediction</h2> <ul> <li>当前先进的序列推荐模型的推理逻辑: <ol> <li>依赖整体序列推断 “general user preferences”</li> <li>格外强调用户最新的交互</li> </ol> </li> </ul> $$ \mathbb{P}\left( v_{t+1}\,|\,v_{t}, v_{t-1}, \ldots, v_1; \Theta \right) \approx \mathbb{P}( v_{t+1}\,|\,v_{t}, \underbrace{\{v_1, v_2, \ldots\}}_{\text{non-sequential}}; \Theta ). $$<div class="slide-highlight"> 💡当前序列推荐模型不约而同地以符合马尔科夫性的方式进行推理！ </div>
<h2 id="short-term--long-term-interests">Short-term & Long-term Interests</h2> <ul> <li><strong>一一对应:</strong></li> </ul> <table> <thead> <tr> <th style="text-align: center">马尔科夫性</th> <th style="text-align: center">推荐理论</th> </tr> </thead> <tbody> <tr> <td style="text-align: center">User Identifiction</td> <td style="text-align: center">Long-term Interest$^{[1]}$</td> </tr> <tr> <td style="text-align: center">Last-Item Attention</td> <td style="text-align: center">Short-term Interest$^{[2]}$</td> </tr> </tbody> </table> <ul> <li><strong>稍有不同:</strong> Long-term interest 的建模并非宣称的那样复杂</li> </ul> <div class="slide-ref"> <div style="width: 100px; height: 1px; background: black; margin-bottom: 5px;"></div> <p style="margin: 2px 0;"> [1] Xie X., et al. Contrastive Learning for Sequential Recommendation. ICDE, 2022. </p> <p style="margin: 2px 0;"> [2] Liu Q., et al. STAMP: Short-term Attention/Memory Priority Model for Session-based Recommendation. KDD, 2018. </p> </div>
<h2 id="rqii-data-for-markovian-reasoner">RQII: Data for Markovian Reasoner</h2> <div class="slide-img"> <img src="https://raw.githubusercontent.com/MTandHJ/blog_source/master/images/20251217120440.png" alt="Image" style="max-width: 100%; height: auto;margin: 0 auto;"> </div>
<h2 id="next-state-prediction">Next-State Prediction</h2> <ul> <li> <p>如何仅凭<strong>上下文</strong>推断马氏链下一时刻状态?</p> $$ s_1, s_2, \ldots, s_t \rightarrow s_{t+1}, \quad s_{n} \in \mathcal{S}, \: \forall n=1,2,\ldots, t+1 $$</li> </ul> <p><strong>Step1:</strong> 根据 $[s_1, s_2, \ldots, s_t]$ 估计转移概率矩阵</p> <p><strong>Step2:</strong> 确定当前时刻的状态 $s_t$</p> <p><strong>Step3:</strong> 选取 $s_t \rightarrow ?$ 最大概率的状态作为预测</p> <ul> <li>擅长 Next-State Prediction 的模型, 需具备: <ol> <li>自适应的序列总结能力;</li> <li>特别着重当前状态的机制</li> </ol> </li> </ul>
<h2 id="markovian-pre-trained-transformer-mpt">Markovian Pre-trained Transformer (MPT)</h2> <ul> <li><strong>Next-State Prediction Task:</strong></li> </ul> $$ \mathcal{L}_{\text{NSP}}(\Theta) = \underset{\mathbf{P} \sim \text{Dir}(\bm{\alpha})}{\mathbb{E}} \underset{\{s_t\}_{t=2}^T \sim \mathbf{P}|s_1}{\mathbb{E}} -\sum_{t=1}^{T-1} \log \mathbb{P}(s_{t+1}\,|\,s_{t}, \ldots, s_{1}; \Theta) $$<div class="slide-highlight"> 100% 人造数据用于预训练！ </div> <div class="slide-highlight"> ✅ Controllable ✅ Unlimited </div>
<h3 id="markovian-pre-training--recommendation-fine-tuning">Markovian Pre-training & Recommendation Fine-tuning</h3> <div class="slide-img"> <img src="https://raw.githubusercontent.com/MTandHJ/blog_source/master/images/20251217143749.png" alt="Image" style="max-width: 100%; height: auto;margin: 0 auto;"> </div>
<h2 id="experiments">Experiments</h2> <div class="slide-img"> <img src="https://raw.githubusercontent.com/MTandHJ/blog_source/master/images/20260112203428.png" alt="Image" style="max-width: 90%; height: auto;margin: 0 auto;"> </div>
<h2 id="data-scaling-ndcg10-vs-tokens">Data Scaling: NDCG@10 vs. #Tokens</h2> <div class="slide-cols"> <div class="slide-col-4"> <div class="slide-img"> <img src="https://raw.githubusercontent.com/MTandHJ/blog_source/master/images/20260112203808.png" alt="Image" style="max-width: 100%; height: auto;margin: 0 auto;"> </div> </div> <div class="slide-col-6"> <ul> <li> <p>$\mathcal{L}_{\text{NSP}}$ 随着 tokens 增加逐渐下降, 且有多次骤降</p> </li> <li> <p>在学习了 $10^{10}$ (约 10B) 左右 tokens 后, 大部分场景下都呈现饱和</p> </li> <li> <p>不同场景下的最优训练 #Tokens 存在差异</p> </li> <li> <p>存在理论上限 Bayes estimator</p> </li> </ul> </div> </div>
<h2 id="comparison-of-inference-mechanisms">Comparison of Inference Mechanisms</h2> <div class="slide-cols"> <div class="slide-col-4"> <ul> <li> <p>MPT 和 Qwen-2.5 的 Backbone 均未经过推荐训练</p> </li> <li> <p>MPT 会更关注自身</p> </li> <li> <p>Qwen-2.5 的 Attention Map 基本上没有区分度</p> </li> <li> <p>MPT 甚至会产生和 SASRec+ 类似的模式</p> </li> </ul> </div> <div class="slide-col-6"> <div class="slide-img"> <img src="https://raw.githubusercontent.com/MTandHJ/blog_source/master/images/20260112204030.png" alt="Image" style="max-width: 100%; height: auto;margin: 0 auto;"> </div> </div> </div>
<h2 id="sensitivity-analysis">Sensitivity Analysis: $|\mathcal{S}|$</h2> <ul> <li>Number of states $|\mathcal{S}|$</li> </ul> <div class="slide-img"> <img src="https://raw.githubusercontent.com/MTandHJ/blog_source/master/images/20251217152317.png" alt="Image" style="max-width: 100%; height: auto;margin: 0 auto;"> </div>
<h2 id="sensitivity-analysis-1">Sensitivity Analysis: $\alpha$</h2> <ul> <li>$\alpha$ of Dirichlet distribution</li> </ul> <div class="slide-img"> <img src="https://raw.githubusercontent.com/MTandHJ/blog_source/master/images/20251217152637.png" alt="Image" style="max-width: 100%; height: auto;margin: 0 auto;"> </div>
<h2 id="summary">Summary</h2> <ul> <li> <p><strong>可迁移的推荐能力:</strong> 序列无关的偏好推断 & 特别关注最新交互</p> </li> <li> <p><strong>Next-State Prediction:</strong> ✅ Controllable ✅ Unlimited</p> </li> <li> <p><strong>Markovian Pre-trained Transformer (MPT):</strong> ✅ 高效 ✅ 易迁移</p> </li> </ul> <div class="slide-highlight"> 下一个时代: Data Simulation? </div>
Thanks!