Journal Article

Research on English semantic modeling and understanding algorithm based on the SemBERT model of natural language processing

Xie, Hengcan, Zhang, Xiaodan, Zhang, Ailing, Zhang, Yongchang

2026

Overview

English machine reading comprehension (MRC) is a pivotal and challenging task in the field of natural language understanding, which requires a model to comprehend a given passage and then answer corresponding questions or infer textual implications. Prevailing deep learning models typically employ word or character levels as fundamental input units. However, the simple concatenation of word and character vectors has been proven suboptimal for semantic representation. To address this limitation, this paper conducts an in-depth investigation and proposes a novel framework centered on fine-grained language unit segmentation and fusion. Our research focuses on hybrid character-word modeling to achieve a more effective mixed-granularity representation. We explore advanced fusion methods for character and word embeddings and introduce a shortlist mechanism based on word frequency filtering, which significantly enhances the training for low-frequency and out-of-vocabulary words. Furthermore, we propose leveraging subwords and specialized uncommon character embeddings to enrich word representations, all within a general subword segmentation framework. Recognizing that many leading MRC models lack genuine semantic understanding and often focus on semantically irrelevant components, we argue that the core of MRC aligns with the goal of semantic role labeling (SRL). Consequently, we innovatively integrate SRL into the MRC and reasoning pipeline to provide richer and more accurate semantic prompts. Extensive experimental evaluations demonstrate that our proposed methods consistently improve the benchmark model across a range of language understanding tasks, including natural language inference, question answering, reading comprehension, semantic similarity, and text classification, achieving competitive performance.

• Publisher: Springer Nature B.V
• Subject: Deep learning / Graphs / Knowledge representation / Language / Large language models / Linguistics / Machine learning / Modelling / Natural language processing / Questions / Reading comprehension / Representations / Segmentation / Semantics / Sentiment analysis / Words (language)