Tags

Addressing Representation Collapse in Vector Quantized Models with One Linear Layer

SimVQ, 坐标变换替代可学习 Codebook

Restructuring Vector Quantization with The Rotation Trick

一种利用 Rotation Trick 来替代 STE 的方案

Is Every Item Worth An Embedding?

是否每个 Item 都值得一个可学习的 Embedding 呢

Let’s Verify Step by Step

来自 OpenAI 的 process supervision

Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations

来自 DeepSeek 的 process supervision

Simba: 符号梯度的可行性

在 Lion 优化器之上的一些探索

Universal Prompt Tuning for Graph Neural Networks

图上特征 prompt 等价各异 graph prompt

All in One: Multi-Task Prompting for Graph Neural Networks

统一 graph/edge/node-level 的 graph prompt

Connection Bottleneck in Attention

Laplace Transform

Laplace 变换的基本概念

Base of RoPE Bounds Context Length

讨论 RoPE base 对于相似 Tokens 感知能力的影响

Round and Round We Go! What makes Rotary Positional Encodings useful?

理解 RoPE 的高低频

Transformers need glasses! Information Over-Squashing in Language Tasks

LLM Representational Collapse

Data Augmentation as Free Lunch: Exploring the Test-Time Augmentation for Sequential Recommendation

TTA, Test-Time Augmentation

1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed

Adam 预训练的 1-bit SGD 优化方法

CPT: Efficient Deep Neural Network Training via Cyclic Precision

CPT, 类似 CosineAnnealingWarmRestarts 的 Precision 循环机制

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

GaLore, 低秩空间中的梯度投影以及权重更新

MICROADAM: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence

MicroAdam, 通过梯度稀疏化以及 error compensation 实现轻量的优化器

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients

Q-GaLore, 对 GaLore 进一步施加低精度量化

SWALP: Stochastic Weight Averaging in Low-Precision Training

SWALP, 通过 SWA 稳定低精度训练

Scaling Laws for Online Advertisement Retrieval

快手, 广告场景下的 Scaling Laws

EMR-MERGING: Tuning-Free High-Performance Model Merging

EMR-MERGING, 多任务权重合并技术

OneRec: Unifying Retrieve and Rank with Generative Recommender and Preference Alignment

OneRec, 端到端的推荐模型

QARM: Quantitative Alignment Multi-Modal Recommendation at Kuaishou

QARM, 多模态推荐对齐与量化

Language Representations Can be What Recommenders Need: Findings and Potentials

Next-token embedding 之于协同过滤

Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation

ReaRec, 多步序列推荐推理

Physics of Language Models: Part 3.1, Knowledge Storage and Extraction

探究 LLM 如何记忆和提取知识的实验性文章

A Self-Attentive Model for Knowledge Tracing

SAKT, 自注意力知识追踪

ECNU 生存指北

ECNU

Dynamic Key-Value Memory Networks for Knowledge Tracing

DKVMN, 带记忆结构的知识追踪

Meta-Learning with Memory-Augmented Neural Networks

MANN, 外置记忆模块

Deep Knowledge Tracing

DKT, 知识追踪

Unifying Generative and Dense Retrieval for Sequential Recommendation

LIGER, 生成式检索的冷启动缺陷与解决方案

Collaborative Alignment for Recommendation

CARec, ID-Textual 特征对齐

Multimodal Pre-training for Sequential Recommendation via Contrastive Learning

MP4SR, 模态融合之于多模态协同过滤

Vector Quantization

SOLO

Autoregressive Image Generation using Residual Quantization

RQ-VAE, 残差向量量化

Recommender Systems with Generative Retrieval

TIGER, 向量量化生成式检索

中研春招聘小记

找工作

Finite Scalar Quantization: VQ-VAE Made Simple

FSQ, 标量量化

Taming Transformers for High-Resolution Image Synthesis

VQGAN, 自回归式的图片生成

Neural Discrete Representation Learning

VQ-VAE, 向量量化的开山之作

Git

Git 的基本操作

Lightweight Optimizers

Multimodal Collaborative Filtering

Network Quantization

Recommendation Foundation Model

Spectral Graph Neural Networks

Zero-shot Composed Image Retrieval