[Paper] EinSort: Sorting is All We Need for Tensorizing LLM

Source: arXiv - 2606.08565v1

Overview

Tensor networks provide efficient representations for compressing large neural networks. By carefully designing shapes and topologies, they can significantly reduce memory and computational costs. However, identifying implicit low-rank structures in large foundation models remains challenging due to their enormous scale and un-structured weight distributions. We propose an adaptive tensorization method that discovers inherent low-rank structure in a target tensor by index ordering. Experiments on weight and KV-cache compression demonstrate improved reconstruction quality compared to baselines.

Key Contributions

This paper presents research in the following areas:

• cs.LG
• cs.AI

Methodology

Please refer to the full paper for detailed methodology.

Practical Implications

This research contributes to the advancement of cs.LG.

Authors

• Toshiaki Koike-Akino
• Jing Liu
• Ye Wang

Paper Information

• arXiv ID: 2606.08565v1
• Categories: cs.LG, cs.AI
• Published: June 7, 2026
• PDF: Download PDF