[Paper] KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection

Source: arXiv - 2603.06552v1

Overview

The KCLarity team tackled SemEval‑2026 Task 6 (CLARITY), which asks systems to spot ambiguous or evasive language in political statements. By experimenting with both direct‑prediction and hierarchical‑label strategies—and even testing zero‑shot large language models—the authors show how modern NLP can help flag political spin before it spreads.

Key Contributions

• Two modeling formulations:
  1. Clarity‑first – predict the “clarity” label directly.
  2. Evasion‑first – predict the finer‑grained “evasion” label and map it to clarity via the task’s taxonomy.
• Encoder‑based baseline: Systematic evaluation of RoBERTa‑large and other transformer encoders on the public test split.
• Zero‑shot decoder‑only experiments: Applied GPT‑5.2 in a pure inference mode (no fine‑tuning) under the evasion‑first formulation.
• Auxiliary training tricks: Explored multi‑task and data‑augmentation setups to boost robustness.
• Empirical insight: Both formulations achieve similar scores, but zero‑shot GPT‑5.2 outperforms fine‑tuned encoders on the hidden evaluation set, hinting at better generalisation.

Methodology

1. Dataset & Taxonomy – The CLARITY task provides political utterances annotated with a binary clarity flag (clear vs. ambiguous) and a multi‑class evasion label (e.g., “hedging”, “deflection”, “vagueness”). The evasion categories are nested under the broader clarity concept.
2. Model families –
   • Encoder‑only: Fine‑tune RoBERTa‑large (and smaller baselines) on the training split, using a standard cross‑entropy loss.
   • Zero‑shot decoder: Prompt GPT‑5.2 with a description of the evasion taxonomy and ask it to label each sentence, without any gradient updates.
3. Formulation switch – For the evasion‑first approach, the predicted evasion class is automatically collapsed to the corresponding clarity label (e.g., any evasion → “ambiguous”).
4. Auxiliary training – Added auxiliary objectives such as next‑sentence prediction and sentiment classification to inject additional linguistic signals.
5. Evaluation – Public test set (known to participants) and a hidden test set (used for final ranking). Metrics: macro‑averaged F1 for clarity and evasion.

Results & Findings

• Encoder vs. Decoder: RoBERTa‑large leads on the public split, but the zero‑shot GPT‑5.2 surpasses it on the hidden split, suggesting better out‑of‑distribution robustness.
• Formulation parity: Direct clarity prediction and the hierarchical evasion‑first route yield almost identical scores, confirming that the taxonomy can be safely leveraged.
• Auxiliary tasks gave a modest 1–2 % boost, especially for low‑resource evasion categories.

Practical Implications

• Fact‑checking pipelines: Integrating an evasion‑first classifier can automatically flag statements that merit deeper human review, reducing the workload for journalists and watchdog NGOs.
• Content moderation: Social‑media platforms can use the model to detect political spin or evasive rhetoric in real time, enabling more transparent policy enforcement.
• Policy‑analysis tools: Researchers building dashboards of political discourse can enrich visualisations with clarity scores, helping citizens spot vague or misleading language.
• Zero‑shot feasibility: The success of GPT‑5.2 shows that large LLMs can be deployed without costly fine‑tuning when the target domain is highly variable, lowering entry barriers for smaller teams.

Limitations & Future Work

• Domain shift: The hidden test set still exposed gaps; models struggled with newly coined political slang and multilingual statements.
• Explainability: Neither encoder nor decoder models provide clear rationales for why a sentence is deemed evasive, limiting trust in high‑stakes settings.
• Taxonomy rigidity: The current hierarchy assumes a fixed set of evasion types; extending it to emerging tactics would require re‑annotation.
• Future directions suggested by the authors include:
  1. Incorporating chain‑of‑thought prompting for LLMs to improve interpretability.
  2. Exploring few‑shot fine‑tuning of decoder models to combine the best of both worlds.
  3. Expanding the dataset to cover non‑English political discourse.

Authors

• Archie Sage
• Salvatore Greco

Paper Information

• arXiv ID: 2603.06552v1
• Categories: cs.CL
• Published: March 6, 2026
• PDF: Download PDF