Journal Article

SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection

Ptaszynski, Michal, Eronen, Juuso, Masui, Fumito, Meléndez, René, Aslan, Lara, Azuma, Daichi

2025

Overview

The automated detection of deceptive language is a crucial challenge in computational linguistics. This study provides a rigorous comparative analysis of three tiers of machine learning models for detecting instructed deception: traditional machine learning (SVM), fine-tuned discriminative models (BERT), and in-context learning with generalist Large Language Models (LLMs). Using the “cross-cultural deception detection” dataset, our findings reveal a clear performance hierarchy. While SVM performance is inconsistent, fine-tuned BERT models achieve substantially superior accuracy. Notably, a multilingual BERT model improves cross-topic accuracy on Spanish text to 90.14%, a gain of over 22 percentage points from its monolingual counterpart (67.20%). In contrast, modern LLMs perform poorly in zero-shot settings and fail to surpass the SVM baseline even with few-shot prompting, underscoring the effectiveness of task-specific fine-tuning. By transparently addressing the limitations of the solicited, low-stakes deception dataset, we establish a robust methodological baseline that clarifies the strengths of different modeling paradigms and informs future research into more complex, real-world deception phenomena.

• Publisher: MDPI AG
• Subject: Accuracy / Architecture / BERT / Cognitive load / Comparative studies / computational linguistics / Datasets / Deception / deception detection / Deep learning / False information / Language / Large language models / Linguistics / Machine learning / multilingual models / Multilingualism / Natural language processing / Neural networks / Product reviews / Propaganda / Semantics / Social networks / Support vector machines