Over-Squashing

Transformers need glasses! Information Over-Squashing in Language Tasks

LLM Representational Collapse