Journal Article

NewsSumm: The World’s Largest Human-Annotated Multi-Document News Summarization Dataset for Indian English

Agarwal, Megha, Agrawal, Avinash, Motghare, Manish

2025

Overview

The rapid growth of digital journalism has heightened the need for reliable multi-document summarization (MDS) systems, particularly in underrepresented, low-resource, and culturally distinct contexts. However, current progress is hindered by a lack of large-scale, high-quality non-Western datasets. Existing benchmarks—such as CNN/DailyMail, XSum, and MultiNews—are limited by language, regional focus, or reliance on noisy, auto-generated summaries. We introduce NewsSumm, the largest human-annotated MDS dataset for Indian English, curated by over 14,000 expert annotators through the Suvidha Foundation. Spanning 36 Indian English newspapers from 2000 to 2025 and covering more than 20 topical categories, NewsSumm includes over 317,498 articles paired with factually accurate, professionally written abstractive summaries. We detail its robust collection, annotation, and quality control pipelines, and present extensive statistical, linguistic, and temporal analyses that underscore its scale and diversity. To establish benchmarks, we evaluate PEGASUS, BART, and T5 models on NewsSumm, reporting aggregate and category-specific ROUGE scores, as well as factual consistency metrics. All NewsSumm dataset materials are openly released via Zenodo. NewsSumm offers a foundational resource for advancing research in summarization, factuality, timeline synthesis, and domain adaptation for Indian English and other low-resource language settings.

• Publisher: MDPI AG
• Subject: abstractive summarization / Annotations / Benchmarks / Datasets / Documents / English language / Hallucinations / human-annotated dataset / Indian English / Linguistics / low-resource languages / Metadata / multi-document summarization / Multilingualism / Narratives / news summarization / Quality control / Reproducibility / Summaries / Taxonomy