[Paper] Toward Global Large Language Models in Medicine

Source: arXiv - 2601.02186v1

Overview

The paper introduces GlobMed, a massive multilingual medical dataset and a benchmark suite that evaluates how well current large language models (LLMs) handle medical tasks across 12 languages—including four low‑resource languages. By training new multilingual medical LLMs (GlobMed‑LLMs) on this data, the authors demonstrate dramatic gains—especially for languages that have historically been left out of AI research—paving the way for more equitable AI‑driven healthcare worldwide.

Key Contributions

• GlobMed dataset: 500 k medical entries covering 12 languages (e.g., English, Spanish, Mandarin, Swahili, Amharic).
• GlobMed‑Bench: A systematic benchmark that tests 56 state‑of‑the‑art LLMs on a variety of multilingual medical tasks (question answering, diagnosis reasoning, summarization, etc.).
• Performance gap analysis: Empirical evidence of large disparities between high‑resource and low‑resource languages in existing models.
• GlobMed‑LLMs: A family of open‑weight multilingual medical LLMs (1.7 B – 8 B parameters) fine‑tuned on GlobMed, achieving >40 % average improvement over baselines and >3× boost for low‑resource languages.
• Open resources: All data, benchmark scripts, and model checkpoints are released publicly to foster community research.

Methodology

1. Data collection & cleaning – The authors aggregated medical texts from public sources (clinical guidelines, research abstracts, patient‑education material) and performed language‑specific preprocessing, de‑duplication, and quality filtering.
2. Benchmark design – Six task categories were defined (e.g., multiple‑choice QA, free‑form diagnosis, clinical note summarization). For each language, balanced test sets were created to ensure comparable difficulty.
3. Model evaluation – 56 LLMs (both proprietary APIs and open‑weight models) were prompted using a unified API. Metrics included accuracy, F1, BLEU/ROUGE for generation, and language‑specific error analysis.
4. Training GlobMed‑LLMs – Existing multilingual base models (e.g., LLaMA‑2, BLOOM) were further fine‑tuned on the GlobMed corpus using a mixture‑of‑experts training schedule that up‑weights low‑resource language data.
5. Statistical analysis – Paired significance tests and regression analyses were used to isolate the impact of multilingual fine‑tuning versus model size.

Results & Findings

• Existing LLMs perform well on English, Mandarin, and Spanish but struggle dramatically on Amharic, Yoruba, and Nepali.
• Fine‑tuning on GlobMed narrows the gap: low‑resource language performance improves by more than threefold, while high‑resource gains are modest but still significant.
• Model size matters, but the multilingual fine‑tuning strategy yields larger relative improvements than simply scaling up parameters.

Practical Implications

• Clinical decision support: Hospitals in low‑resource regions can deploy GlobMed‑LLMs for triage chatbots, symptom checkers, or medical record summarization in local languages, reducing reliance on English‑only tools.
• Medical education: Multilingual study aids and question banks can be generated automatically, supporting curricula in under‑represented languages.
• Regulatory compliance: By providing transparent, open‑weight models, developers can audit and adapt the models to meet local data‑privacy laws (e.g., GDPR, HIPAA equivalents).
• Rapid prototyping: The benchmark suite lets product teams quickly evaluate whether an off‑the‑shelf LLM meets the linguistic requirements of their target market before committing to costly fine‑tuning.
• Research acceleration: Open data and evaluation scripts lower the barrier for academic and industry groups to explore multilingual health AI, fostering competition and innovation.

Limitations & Future Work

• Domain coverage: While extensive, GlobMed still leans heavily toward publicly available literature; rare disease case reports and non‑textual data (e.g., imaging) are under‑represented.
• Cultural nuance: The benchmark focuses on factual correctness but does not fully capture culturally appropriate communication styles, which are crucial for patient‑facing applications.
• Model size ceiling: Experiments capped at 8 B parameters; scaling to >50 B may reveal different trade‑offs, especially for high‑resource languages.
• Evaluation breadth: Real‑world deployment studies (e.g., user studies with clinicians in low‑resource settings) are needed to validate safety and usability.

The authors plan to expand GlobMed with more languages, incorporate multimodal medical data, and launch a community‑driven “challenge” to stimulate the next generation of equitable medical AI.

Authors

• Rui Yang
• Huitao Li
• Weihao Xuan
• Heli Qi
• Xin Li
• Kunyu Yu
• Yingjian Chen
• Rongrong Wang
• Jacques Behmoaras
• Tianxi Cai
• Bibhas Chakraborty
• Qingyu Chen
• Lionel Tim‑Ee Cheng
• Marie‑Louise Damwanza
• Chido Dzinotyiwei
• Aosong Feng
• Chuan Hong
• Yusuke Iwasawa
• Yuhe Ke
• Linah Kitala
• Taehoon Ko
• Jisan Lee
• Irene Li
• Jonathan Chong Kai Liew
• Hongfang Liu
• Lian Leng Low
• Edison Marrese‑Taylor
• Yutaka Matsuo
• Isheanesu Misi
• Yilin Ning
• Jasmine Chiat Ling Ong
• Marcus Eng Hock Ong
• Enrico Petretto
• Hossein Rouhizadeh
• Abiram Sandralegar
• Oren Schreier
• Iain Bee Huat Tan
• Patrick Tan
• Daniel Shu Wei Ting
• Junjue Wang
• Chunhua Weng
• Matthew Yu Heng Wong
• Fang Wu
• Yunze Xiao
• Xuhai Xu
• Qingcheng Zeng
• Zhuo Zheng
• Yifan Peng
• Douglas Teodoro
• Nan Liu

Paper Information

• arXiv ID: 2601.02186v1
• Categories: cs.CL
• Published: January 5, 2026
• PDF: Download PDF