Journal Article

Identification of Emerging Technology Topics and Prediction of Trends Using a Method Integrating BERTopic and IWOA-BiLSTM Models

CHEN, Yuanyuan, FU Bin, GAO, Yuan, QIAO Junwei

2025

Overview

[Purpose/Significance] With the rapid advancement of global science and technology, emerging technologies are constantly evolving, placing higher demands on national strategic planning and resource allocation. Artificial intelligence (AI), as a core driver of the current technological revolution, requires close attention to its internal technical topic evolution to anticipate disruptive changes and guide the direction innovation. Although existing research primarily focuses on identifying technical topics through bibliometric or patent analysis, there is still insufficient quantitative forecasting of their future development. To address this gap, this study proposes an integrated analytical framework that combines BERTopic-based topic modeling with an IWOA-optimized BiLSTM neural network, achieving a unified approach to both topic identification and trend forecasting. Unlike traditional LDA models or expert-based subjective judgment, this method demonstrates significant advancements in semantic representation, model optimization, and prediction accuracy. It expands the theoretical boundaries of emerging technology forecasting and offers valuable quantitative support for science and technology policy-making. [Method/Process] This study utilized 22,243 AI-related patent records collected from 2015 to 2024. BERTopic was applied to extract representative technology topics from patent abstracts. A multi-dimensional evaluation system was constructed using three indicators: novelty, impact, and growth rate, capturing different aspects of emerging technologies. The CRITIC method was employed to objectively assign weights to each dimension, enhancing the robustness and balance of the composite index. BERTopic, which integrates BERT-based semantic embeddings with HDBSCAN density-based clustering, improves both the coherence and granularity of topic extraction. For trend prediction, an Improved Whale Optimization Algorithm (IWOA) was introduced to fine-tune BiLSTM's learning rate, epoch count, and hidden layer size. IWOA enhances global optimization through Gaussian chaos initialization, Levy flight strategy, nonlinear control factors, and elite reverse learning. [Results/Conclusions] Experimental results show that BERTopic achieves superior topic coherence compared to baseline models and successfully identifies five emerging technical areas, including Intelligent Models and Algorithms, Information Processing, Deep Neural Networks, Smart Robotics, and Numerical Control Systems. The IWOA-BiLSTM model outperforms conventional LSTM and BiLSTM models in error metrics (MAPE, RMSE, and MAE), confirming its predictive advantage. Forecast results indicate that these emerging topics will experience sustained growth over the next five years, reflecting strong application potential and industrial value. This study confirms the feasibility and effectiveness of the integrated \"identification–prediction\" framework, providing a data-driven tool for strategic decision-making in science and technology development. Limitations include dependence on data quality and a current focus on the field of AI. Future research should expand the framework to other strategic areas, such as renewable energy, biomedicine, and intelligent manufacturing, to further validate its generalizability.

• Publisher: Agricultural Information Institute of Chinese Academy of Agricultural Sciences,Editorial Department of Journal of Library and Information Science in Agriculture
• Subject: emerging technologies|topic identification trend prediction|bertopic|iwoa-bilstm