[Paper] When AI Writes, Whose Voice Remains? Quantifying Cultural Marker Erasure Across World English Varieties in Large Language Models

Source: arXiv - 2602.22145v1

Overview

The paper “When AI Writes, Whose Voice Remains? Quantifying Cultural Marker Erasure Across World English Varieties in Large Language Models” investigates a subtle but important bias in modern LLMs: when they rewrite or “professionalize” text, they often strip away linguistic features that signal a speaker’s cultural identity (e.g., Indian, Singaporean, or Nigerian English). The authors coin the term “Cultural Ghosting” to describe this systematic erasure and provide the first quantitative metrics to measure it.

Key Contributions

• Definition of Cultural Ghosting: Introduces a clear, research‑grade concept for the loss of culturally specific language markers in LLM‑generated text.
• Two novel metrics:
  • Identity Erasure Rate (IER) – proportion of cultural markers removed by the model.
  • Semantic Preservation Score (SPS) – semantic similarity between original and rewritten text (based on embeddings).
• Large‑scale empirical study: Analyzes 22,350 outputs from five popular LLMs (including GPT‑3.5, Claude, LLaMA, etc.) across three prompt styles and three World English varieties (Indian, Singaporean, Nigerian).
• Discovery of the “Semantic Preservation Paradox”: Models can retain meaning (high SPS) while still erasing cultural cues.
• Mitigation experiment: Shows that a simple “preserve cultural markers” instruction cuts IER by ~29 % without hurting SPS.

Methodology

1. Dataset construction – The authors curated 1,490 sentences that contain culturally marked lexical items (e.g., “lah”, “cheer up” used as politeness particles) and pragmatic conventions typical of the three target English varieties.
2. Prompt conditions – Each sentence was fed to five LLMs under three prompts:
   • Neutral rewrite (no extra instruction)
   • Professional tone (e.g., “Make this sound formal”)
   • Cultural‑preservation (explicitly ask the model to keep original cultural markers).
3. Marker detection – A rule‑based + statistical tagger identifies cultural markers in both source and generated texts.
4. Metric calculation –
   • IER = (Number of markers removed) / (Total markers in source).
   • SPS = cosine similarity between sentence embeddings (SBERT) of source and output.
5. Statistical analysis – ANOVA and post‑hoc tests assess differences across models, prompts, and marker types (lexical vs. pragmatic).

Results & Findings

• Pragmatic markers (politeness particles, discourse markers) are erased 1.9× more often than purely lexical markers (71.5 % vs. 37.1 %).
• Semantic Preservation Paradox: despite a near‑perfect semantic match, the cultural “voice” is often lost.
• Prompt impact: The cultural‑preservation prompt reduces IER by ≈29 % (e.g., from 10.3 % to 7.3 %) while SPS stays essentially unchanged (Δ ≈ 0.01).
• Model variance: Smaller, instruction‑tuned models tend to erase more markers, whereas larger, more diverse models (e.g., GPT‑4) show the lowest IER.

Practical Implications

• Product design: Any SaaS that auto‑rewrites emails, reports, or chat messages should expose a “preserve cultural style” toggle to avoid unintentionally homogenizing user voice.
• Developer tooling: LLM‑based code‑comment generators, documentation assistants, and knowledge‑base summarizers need to be aware that “cleaning up” text can strip regional identity, potentially alienating non‑native speakers.
• Compliance & DEI: Companies with global workforces can use the IER metric as a diagnostic to audit internal AI pipelines for cultural bias, supporting diversity‑equity‑inclusion goals.
• Fine‑tuning strategies: Adding culturally marked examples to the instruction‑tuning set, or employing a post‑processing filter that re‑injects detected markers, can mitigate ghosting without sacrificing meaning.
• User experience: Retaining culturally specific politeness conventions can improve perceived politeness and trust in AI‑generated communications, especially in customer‑facing contexts (e.g., chatbots for Indian or Nigerian markets).

Limitations & Future Work

• Scope of varieties: The study focuses on three World English varieties; results may differ for other dialects (e.g., Caribbean, Malaysian English).
• Marker detection reliance: The rule‑based tagger may miss subtle or emergent cultural markers, potentially under‑estimating IER.
• Semantic metric: SPS uses static sentence embeddings; more nuanced meaning preservation (e.g., discourse coherence) could be explored.
• Mitigation depth: The “preserve cultural markers” prompt is a blunt instrument; future work could investigate fine‑grained control tokens or adapter layers that explicitly model cultural style.
• Human evaluation: While automatic metrics are informative, large‑scale human judgments would solidify claims about perceived voice loss and acceptability.

Authors

• Satyam Kumar Navneet
• Joydeep Chandra
• Yong Zhang

Paper Information

• arXiv ID: 2602.22145v1
• Categories: cs.HC, cs.AI, cs.CL
• Published: February 25, 2026
• PDF: Download PDF