[Paper] TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning

Source: arXiv - 2606.06494v1

Overview

Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed reference frame to learn a low-rank update applied to the singular value matrix. A soft spectral penalty discourages updates aligned with dominant singular directions, reducing interference while routing fine-grained adaptation into the highly flexible, long-tail spectral coordinates.

Key Contributions

This paper presents research in the following areas:

• cs.LG

Methodology

Please refer to the full paper for detailed methodology.

Practical Implications

This research contributes to the advancement of cs.LG.

Authors

• Marius Dragoi
• Ioana Pintilie
• Alexandra Dragomir
• Antonio Barbalau
• Florin Brad

Paper Information

• arXiv ID: 2606.06494v1
• Categories: cs.LG
• Published: June 4, 2026
• PDF: Download PDF