Tags

Pareto Multi-Task Learning

通过限定子区域实现 Pareto MTL

Multiple-Gradient Descent Algorithm (MGDA) for Multiobjective Optimization

从梯度融合角度理解多任务/多目标优化

Product Quantization for Nearest Neighbor Search

将 Product Quantization 用于搜索以降低存储占用

Bridging Textual-Collaborative Gap through Semantic Codes for Sequential Recommendation

通过 Q-Former 将 semantic IDs 转换为 textual IDs

An Image is Worth 32 Tokens for Reconstruction and Generation

非结构化的一维离散 Visual Tokens 生成策略

BEiT: BERT Pre-Training of Image Transformers

Visual Tokens & Masked Image Modeling

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

自回归图像生成: Multi-scale Quantization & Next-scale Prediction

TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation

兼顾 Low-level 的 Pixel 信息和 High-level 的 Semantic 信息

UniCode$^2$: Cascaded Large-scale Codebooks for Unified Multimodal Understanding and Generation

非常自然的 Image-Codeword+Text-LLM-NextCodeWord-Generation 流程

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling

符号序列的自动切分, 探究非 Subword Tokenizer 的可能性

SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer

TiTok 框架 + Softmax 版 Vector Quantization 以期更高的压缩比

MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers

多尺度 Transformer, 探究非 Subword Tokenizer 的可能性

SpaceByte: Towards Deleting Tokenization from Large Language Modeling

探究非 Subword Tokenizer 的可能性

Is Vector Quantization the Future of Recommendation?

Temporal Cross-Effects in Knowledge Tracing

借鉴 Hawkes Process 思想建模知识掌握的交叉影响和衰减

EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register Tokens

通过减小 KV cache size 加速 LLMRec 的推理

Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations

COBRA, 链接离散编码和稠密表示的尝试

Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation

LC-Rec, LLM + RQ-VAE + 丰富的多任务训练

Sinkhorn Distance and Sinkhorn-Knopp Algorithm

关于利用 Sinkhorn 距离求解离散最优传输问题的记录

OneRec: Unifying Retrieve and Rank with Generative Recommender and Preference Alignment

OneRec, 端到端的推荐模型

SEvo

Addressing Representation Collapse in Vector Quantized Models with One Linear Layer

SimVQ, 坐标变换替代可学习 Codebook

Restructuring Vector Quantization with The Rotation Trick

一种利用 Rotation Trick 来替代 STE 的方案

Is Every Item Worth An Embedding?

是否每个 Item 都值得一个可学习的 Embedding 呢

Let’s Verify Step by Step

来自 OpenAI 的 process supervision

Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations

来自 DeepSeek 的 process supervision

Simba: 符号梯度的可行性

在 Lion 优化器之上的一些探索

环境配置

个人的环境配置

Universal Prompt Tuning for Graph Neural Networks

图上特征 prompt 等价各异 graph prompt

All in One: Multi-Task Prompting for Graph Neural Networks

统一 graph/edge/node-level 的 graph prompt

Connection Bottleneck in Attention

Laplace Transform

Laplace 变换的基本概念

Base of RoPE Bounds Context Length

讨论 RoPE base 对于相似 Tokens 感知能力的影响

Round and Round We Go! What makes Rotary Positional Encodings useful?

理解 RoPE 的高低频

Transformers need glasses! Information Over-Squashing in Language Tasks

LLM Representational Collapse

Data Augmentation as Free Lunch: Exploring the Test-Time Augmentation for Sequential Recommendation

TTA, Test-Time Augmentation

1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed

Adam 预训练的 1-bit SGD 优化方法

CPT: Efficient Deep Neural Network Training via Cyclic Precision

CPT, 类似 CosineAnnealingWarmRestarts 的 Precision 循环机制

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

GaLore, 低秩空间中的梯度投影以及权重更新

MICROADAM: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence

MicroAdam, 通过梯度稀疏化以及 error compensation 实现轻量的优化器

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients

Q-GaLore, 对 GaLore 进一步施加低精度量化

SWALP: Stochastic Weight Averaging in Low-Precision Training

SWALP, 通过 SWA 稳定低精度训练

Scaling Laws for Online Advertisement Retrieval

快手, 广告场景下的 Scaling Laws

EMR-MERGING: Tuning-Free High-Performance Model Merging

EMR-MERGING, 多任务权重合并技术

QARM: Quantitative Alignment Multi-Modal Recommendation at Kuaishou

QARM, 多模态推荐对齐与量化

Language Representations Can be What Recommenders Need: Findings and Potentials

Next-token embedding 之于协同过滤

Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation

ReaRec, 多步序列推荐推理

Physics of Language Models: Part 3.1, Knowledge Storage and Extraction

探究 LLM 如何记忆和提取知识的实验性文章

A Self-Attentive Model for Knowledge Tracing

SAKT, 自注意力知识追踪

ECNU 生存指北

ECNU

Dynamic Key-Value Memory Networks for Knowledge Tracing

DKVMN, 带记忆结构的知识追踪

Meta-Learning with Memory-Augmented Neural Networks

MANN, 外置记忆模块

Deep Knowledge Tracing

DKT, 知识追踪

Unifying Generative and Dense Retrieval for Sequential Recommendation

LIGER, 生成式检索的冷启动缺陷与解决方案

Collaborative Alignment for Recommendation

CARec, ID-Textual 特征对齐

Multimodal Pre-training for Sequential Recommendation via Contrastive Learning

MP4SR, 模态融合之于多模态协同过滤

SOLO

Autoregressive Image Generation using Residual Quantization

RQ-VAE, 残差向量量化

Recommender Systems with Generative Retrieval

TIGER, 向量量化生成式检索

中研春招聘小记

找工作

Finite Scalar Quantization: VQ-VAE Made Simple

FSQ, 标量量化

Taming Transformers for High-Resolution Image Synthesis

VQGAN, 自回归式的图片生成

Neural Discrete Representation Learning

VQ-VAE, 向量量化的开山之作

Git

Git 的基本操作

Deep Knowledge Tracing

Lightweight Optimizers

Multimodal Collaborative Filtering

Network Quantization

Recommendation Foundation Model

Spectral Graph Neural Networks

Tokenization is the Language of AI

Zero-shot Composed Image Retrieval