[Paper] CLEF HIPE-2026: Evaluating Accurate and Efficient Person-Place Relation Extraction from Multilingual Historical Texts

Source: arXiv - 2602.17663v1

Overview

The CLEF HIPE‑2026 lab pushes the frontier of historical text mining by challenging systems to automatically discover who was where across noisy, multilingual documents spanning centuries. By extending earlier HIPE campaigns, the authors introduce a realistic benchmark that blends linguistic nuance, temporal reasoning, and computational efficiency—key ingredients for building robust digital‑humanities pipelines and next‑generation knowledge graphs.

Key Contributions

• New multilingual benchmark for person‑place relation extraction covering several languages and historical periods.
• Two nuanced relation types:
  • at – “Has the person ever been at this place?”
  • isAt – “Is the person located at this place around the publication time?”
• Three‑dimensional evaluation that simultaneously measures:
  1. Accuracy (precision/recall on relation classification)
  2. Computational efficiency (runtime & resource usage)
  3. Domain generalization (performance on unseen time slices or corpora)
• Open‑source data & tooling for reproducible experiments, including pre‑annotated corpora, temporal‑geographic grounding resources, and baseline implementations.
• Clear link to downstream applications such as knowledge‑graph enrichment, biographical reconstruction, and spatial analytics for historians.

Methodology

1. Corpus Construction – The organizers gathered a heterogeneous set of historical texts (newspapers, letters, archival reports) in languages like English, German, French, and Italian. Automatic OCR pipelines were applied, preserving the typical noise (misspellings, layout artifacts) found in digitized archives.
2. Annotation Scheme – Human annotators labeled person entities, place entities, and the two relation types, also providing temporal tags (e.g., “c. 1850”, “post‑World‑War II”). This yields a multilabel, temporally‑aware dataset.
3. Task Definition – Participants receive raw text and a list of candidate person/place pairs. The system must:
   • Detect whether a relation exists (at or isAt).
   • Output a confidence score for each predicted relation.
4. Evaluation Framework –
   • Accuracy: micro‑averaged F1 across both relation types.
   • Efficiency: measured wall‑clock time and memory on a standard benchmark machine.
   • Generalization: participants are tested on a held‑out historical period and on a language not seen during training.
5. Baseline Systems – The paper presents a strong baseline that combines multilingual BERT embeddings, a temporal reasoning layer (rule‑based date matching), and a lightweight graph‑based post‑processor to enforce geographic consistency.

Results & Findings

• Top‑performing systems achieved F1 ≈ 78 % on the at relation and F1 ≈ 71 % on isAt, showing that modern multilingual transformers can handle noisy historical language when coupled with temporal cues.
• Efficiency trade‑offs were evident: the highest‑accuracy models required 2–3× more GPU memory and longer inference times, while a compact BiLSTM‑CRF baseline ran in real‑time but lagged by ~10 % in F1.
• Generalization scores dropped by ~12 % when moving to an unseen language (Italian), highlighting the need for better cross‑lingual transfer techniques.
• Error analysis revealed that most mistakes stemmed from ambiguous temporal expressions (“the early 1900s”) and place name disambiguation (e.g., multiple towns named “Springfield”).

Practical Implications

• Knowledge‑Graph Construction – Automated person‑place links can enrich historical KG projects (e.g., DBpedia Historical, Europeana) without manual curation, accelerating research on social networks of the past.
• Digital Biography Tools – Genealogy platforms and scholarly biography editors can auto‑populate timelines with verified location stamps, reducing the labor‑intensive fact‑checking step.
• Spatial Humanities Analytics – Researchers can generate heat‑maps of migration patterns, trade routes, or cultural diffusion directly from text corpora, enabling new visualizations and hypothesis testing.
• Multilingual Heritage Apps – Museums and cultural institutions can power multilingual exhibit guides that dynamically surface “where was this figure active?” based on underlying archival texts.
• Efficiency‑aware Deployments – The three‑fold evaluation encourages developers to balance model size with speed, making it feasible to run extraction pipelines on modest cloud instances or even on‑premise archival servers.

Limitations & Future Work

• Temporal Granularity – Current annotations operate at a coarse decade‑century level; finer‑grained dating (exact years or months) remains an open challenge.
• Place Disambiguation – The benchmark does not yet integrate a robust gazetteer linking to modern geocodes, limiting downstream GIS applications.
• Cross‑lingual Transfer – Performance on unseen languages suggests that multilingual pre‑training alone is insufficient; future work could explore adapter modules or few‑shot learning.
• Scalability – While efficiency is measured, scaling to hundreds of millions of documents (e.g., national newspaper archives) will require further engineering, such as streaming inference or model quantization.

Overall, HIPE‑2026 offers a realistic, well‑rounded testbed that bridges academic research and real‑world needs in the digital humanities, giving developers a concrete target for building next‑generation, multilingual historical information extraction systems.

Authors

• Juri Opitz
• Corina Raclé
• Emanuela Boros
• Andrianos Michail
• Matteo Romanello
• Maud Ehrmann
• Simon Clematide

Paper Information

• arXiv ID: 2602.17663v1
• Categories: cs.AI, cs.CL, cs.IR
• Published: February 19, 2026
• PDF: Download PDF