Journal Article

DravidianCodeMix: sentiment analysis and offensive language identification dataset for Dravidian languages in code-mixed text

Suryawanshi, Shardul, McCrae, John P, Chakravarthi, Bharathi Raja, Muralidaran, Vigneshwaran, Sherly, Elizabeth, Priyadharshini, Ruba, Jose, Navya

2022

Overview

This paper describes the development of a multilingual, manually annotated dataset for three under-resourced Dravidian languages generated from social media comments. The dataset was annotated for sentiment analysis and offensive language identification for a total of more than 60,000 YouTube comments. The dataset consists of around 44,000 comments in Tamil-English, around 7000 comments in Kannada-English, and around 20,000 comments in Malayalam-English. The data was manually annotated by volunteer annotators and has a high inter-annotator agreement in Krippendorff’s alpha. The dataset contains all types of code-mixing phenomena since it comprises user-generated content from a multilingual country. We also present baseline experiments to establish benchmarks on the dataset using machine learning and deep learning methods. The dataset is available on Github and Zenodo.

• Publisher: Springer Nature B.V
• Subject: Archaeology / Bilingualism / Code switching / Comments / Computer mediated communication / Data mining / Datasets / Deep learning / Dravidian languages / English language / Experiments / Identification / Internet / Kannada language / Language / Language identification / Languages / Machine learning / Malayalam / Mass media / Multilingualism / Natural language / Presidential elections / Product reviews / Reputation management / Scripts / Sentiment analysis / Social media / Social networks / Tamil language / User generated content / Verbal communication