[Paper] PONTE: Personalized Orchestration for Natural Language Trustworthy Explanations

Source: arXiv - 2603.06485v1

Overview

The paper introduces PONTE, a human‑in‑the‑loop framework that generates natural‑language explanations for AI models while tailoring the style and content to individual users. By treating personalization as a closed‑loop validation and adaptation process—rather than a static prompt—it aims to deliver explanations that are both trustworthy (faithful to the underlying model) and aligned with each user’s expertise, goals, and cognitive preferences.

Key Contributions

• Closed‑loop personalization: A feedback‑driven loop that updates a low‑dimensional user‑preference model instead of relying on brittle prompt engineering.
• Preference‑conditioned generator: A language model that produces explanations grounded in structured XAI artifacts (e.g., feature importance tables, decision rules) while being steered by the learned preference vector.
• Multi‑facet verification modules: Automatic checks for numerical faithfulness, informational completeness, and stylistic alignment; optional retrieval‑grounded argumentation to enrich explanations.
• Empirical validation: Experiments on healthcare (diagnostic risk scores) and finance (credit‑scoring models) showing significant gains in completeness and style conformity compared with a baseline that generates without verification.
• Human‑in‑the‑loop study: Demonstrates that users can reliably convey their stylistic needs, that the system’s output matches those needs, and that quality ratings remain high even when generation randomness is introduced.

Methodology

1. Preference Modeling – Each user is represented by a compact vector (≈10 dimensions) encoding stylistic desiderata such as formality, technical depth, visual vs. textual emphasis, and risk tolerance. The vector is initialized from a short questionnaire or a few interaction examples.
2. Grounded Generation – A large language model (LLM) receives two inputs: (a) the structured XAI artifact (e.g., SHAP values, rule sets) and (b) the current preference vector. The LLM is fine‑tuned to condition its output on the vector, producing a natural‑language narrative that explains the artifact.
3. Verification Loop – After generation, three automatic validators run:
   • Faithfulness: Checks that numeric statements (e.g., “feature X contributed 0.23”) match the source artifact.
   • Completeness: Ensures all required components (e.g., top‑k features, confidence intervals) appear.
   • Stylistic Alignment: Measures similarity between the generated text’s style metrics (lexical richness, sentence length, jargon density) and the target preference vector.
     If any validator fails, the system either (i) edits the text automatically or (ii) prompts the user for clarification, then updates the preference vector accordingly.
4. Iterative Refinement – The loop repeats until all validators pass or a maximum iteration budget is reached, yielding a personalized, trustworthy explanation.

Results & Findings

• Quantitative gains: Across both domains, the verification‑refinement loop increased completeness scores by ~22 % and reduced faithfulness violations by ~35 % relative to a generation‑only baseline.
• Stylistic fidelity: Cosine similarity between intended and realized preference vectors rose from 0.58 (baseline) to 0.86 after two feedback cycles.
• Robustness to stochasticity: Even when the underlying LLM sampled diverse outputs, the verification modules consistently filtered out style‑drifted or fact‑inconsistent drafts.
• Human study: 48 participants (mix of clinicians, data scientists, and business analysts) rated PONTE explanations as “clear” (4.6/5) and “trustworthy” (4.4/5), with 92 % indicating the style matched their expectations.

Practical Implications

• Developer toolkits: PONTE can be wrapped as an API that accepts model‑agnostic XAI artifacts and a user‑profile payload, returning ready‑to‑display explanations for dashboards, chat‑bots, or compliance reports.
• Regulatory compliance: Financial and healthcare firms can meet “right‑to‑explain” mandates while respecting diverse stakeholder needs (e.g., a regulator vs. a patient).
• Customer‑facing AI products: SaaS platforms can let end‑users toggle explanation depth (high‑level summary vs. technical breakdown) without engineering a new prompt for each setting.
• Reduced support overhead: By automatically detecting and correcting faithfulness errors, PONTE lowers the risk of misleading explanations that would otherwise require manual QA.

Limitations & Future Work

• Preference capture overhead: Initial preference elicitation still needs a brief questionnaire; scaling to thousands of users may require smarter implicit profiling (e.g., interaction logs).
• Domain‑specific artifacts: The current implementation assumes structured XAI outputs (feature importance, rule sets). Extending to black‑box explanations like counterfactuals or visual saliency maps will need additional grounding strategies.
• Verification scalability: Faithfulness checks rely on exact numeric matching, which can be brittle for models that produce stochastic scores; future work could incorporate probabilistic verification.
• User study breadth: Experiments were limited to two domains and a modest participant pool; broader field trials (e.g., autonomous driving, legal AI) are needed to validate generality.

PONTE marks a step toward AI explanations that are not only accurate but also personally meaningful—an essential capability as intelligent systems become ever more embedded in everyday decision‑making.

Authors

• Vittoria Vineis
• Matteo Silvestri
• Lorenzo Antonelli
• Filippo Betello
• Gabriele Tolomei

Paper Information

• arXiv ID: 2603.06485v1
• Categories: cs.CL, cs.AI
• Published: March 6, 2026
• PDF: Download PDF