[Paper] Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research

Source: arXiv - 2606.12247v1

Overview

Research on bias in large language models (LLMs) has predominantly focused on third-person audits, which study how models represent or evaluate demographic groups as external subjects. However, this paradigm overlooks a structural blind spot because the user is absent from the audit. In practice, LLMs are used in open-ended, personal interactions, during which the model implicitly represents the user and adjusts its responses accordingly. When identical requests yield different responses depending on who is asking, bias manifests not in how the model describes others but in how it treats its interlocutor. We propose Situated Interaction Auditing (SIA), a user-centered framework for studying how user profile signals — implicit sociodemographic markers, writing style, and stated identity — systematically shape LLM response quality, content, and tone. We demonstrate the framework through a case study that intersects gender and socioeconomic status signals across multiple task domains and outline a research agenda for SIA as a new mission for natural language processing.

Key Contributions

This paper presents research in the following areas:

• cs.CY
• cs.CL

Methodology

Please refer to the full paper for detailed methodology.

Practical Implications

This research contributes to the advancement of cs.CY.

Authors

• Andrés Abeliuk
• Cinthia Sanchez Macias
• Valentina Alarcón
• Álvaro Madariaga
• Claudia Lopez

Paper Information

• arXiv ID: 2606.12247v1
• Categories: cs.CY, cs.CL
• Published: June 10, 2026
• PDF: Download PDF