Explore the vast range of titles available.

Significant wave height (SWH) is a key parameter for monitoring the state of waves. Accurate and long-term SWH forecasting is significant to maritime shipping and coastal engineering. This study proposes a transformer model based on an attention mechanism to achieve the forecasting of SWHs. The transformer model can capture the contextual information and dependencies between sequences and achieves continuous time series forecasting. Wave scale classification is carried out according to the forecasting results, and the results are compared with gated recurrent unit (GRU) and long short-term memory (LSTM) machine-learning models and the key laboratory of MArine Science and NUmerical Modeling (MASNUM) numerical wave model. The results show that the machine-learning models outperform the MASNUM within 72 h, with the transformer being the best model. For continuous 12 h, 24 h, 36 h, 48 h, 72 h, and 96 h forecasting, the average mean absolute errors (MAEs) of the test sets were, respectively, 0.139 m, 0.186 m, 0.223 m, 0.254 m, 0.302 m, and 0.329 m, and the wave scale classification accuracies were, respectively, 91.1%, 99.4%, 86%, 83.3%, 78.9%, and 77.5%. The experimental results validate that the transformer model can achieve continuous and accurate SWH forecasting, as well as accurate wave scale classification and early warning of waves, providing technical support for wave monitoring.

Ship trajectory prediction is a complex time series forecasting problem that necessitates models capable of accurately capturing both long-term trends and short-term fluctuations in vessel movements. While existing deep learning models excel in short-term predictions, they struggle with long-sequence time series forecasting (LSTF) due to difficulties in capturing long-term dependencies, resulting in significant prediction errors. This paper proposes the Informer-TP method, leveraging Automatic Identification System (AIS) data and based on the Informer model, to enhance the ability to capture long-term dependencies, thereby improving the accuracy of long-term ship trajectory predictions. Firstly, AIS data are preprocessed and divided into trajectory segments. Secondly, the time series is separated from the trajectory data in each segment and input into the model. The Informer model is utilized to improve long-term ship trajectory prediction ability, and the output mechanism is adjusted to enable predictions for each segment. Finally, the proposed model’s effectiveness is validated through comparisons with baseline models, and the influence of various sequence lengths Ltoken on the Informer-TP model is explored. Experimental results show that compared with other models, the proposed model exhibits the lowest Mean Squared Error, Mean Absolute Error, and Haversine distance in long-term forecasting, demonstrating that the model can effectively capture long-term dependencies in the trajectories, thereby improving the accuracy of long-term vessel trajectory predictions. This provides an effective and feasible method for ensuring ship navigation safety and advancing intelligent shipping.

Accurate long‐sequence net load forecasting is essential for reliable grid operation and renewable integration, yet it remains challenging under quasi‐periodicity, sharp weather‐driven variability and long‐range error accumulation. We propose HarmoNet, an end‐to‐end dual‐domain architecture for long‐horizon deterministic and interval forecasting. HarmoNet (i) encodes coupled low/high‐frequency representations with multi‐scale temporal signals, (ii) integrates local pattern modelling and global dependency learning via a hybrid convolutional‐transformer block and (iii) aggregates horizon‐wide representations to mitigate drift in long‐range prediction. Uncertainty is estimated with quantile regression (10%–50%–90%). We evaluate HarmoNet on four‐year hourly net‐load datasets from Belgium, Bulgaria and Italy (2016–2019) derived from the Open Power System Data platform, using eight exogenous meteorological covariates, over horizons of 96/192/336/720 h. Relative to the strongest baseline per dataset‐horizon setting, HarmoNet reduces MAE by 14.2% on average (up to 22.5% on Italy at 720 h) and achieves average reductions of 28.4% in the Winkler score and 13.0% in pinball loss. Under deployment‐oriented stress tests spanning high‐volatility, peak‐spike and steep‐ramp windows, HarmoNet attains the best deterministic accuracy in 27/36 windows and the best probabilistic performance in 32/36 windows, indicating robust and deployment‐friendly long‐horizon forecasting.

Using optimal assembly relationships, companies can enhance product quality without significantly increasing production costs. However, predicting Assembly Geometric Errors presents a challenging real-world problem in the manufacturing domain. To address this challenge, this paper introduces a highly efficient Transformer-based neural network model known as Predicting Assembly Geometric Errors based on Transformer (PAGEformer). This model accurately captures long-range assembly relationships and predicts final assembly errors. The proposed model incorporates two unique features: firstly, an enhanced self-attention mechanism to more effectively handle long-range dependencies, and secondly, the generation of positional information regarding gaps and fillings to better capture assembly relationships. This paper collected actual assembly data for folding rudder blades for unmanned aerial vehicles and established a Mechanical Assembly Relationship Dataset (MARD) for a comparative study. To further illustrate PAGEformer performance, we conducted extensive testing on a large-scale dataset and performed ablation experiments. The experimental results demonstrated a 15.3% improvement in PAGEformer accuracy compared to ARIMA on the MARD. On the ETH, Weather, and ECL open datasets, PAGEformer accuracy increased by 15.17%, 17.17%, and 9.5%, respectively, compared to the mainstream neural network models.

Human gait trajectory prediction is a long-standing research topic in human–machine interaction. However, there are two shortcomings in the current gait trajectory prediction technology. The first shortcoming is that the neural network model of gait prediction only predicts dozens of future time frames of gait trajectory. The second shortcoming is that the gait prediction neural network model is uninterpretable. We propose the Interpretable-Concatenation former (IC-former) model, which can predict long-term gait trajectories and explain the prediction results by quantifying the importance of data at different positions in the input sequence. Experiments prove that the IC-former model we proposed not only makes a breakthrough in prediction accuracy but also successfully explains the data basis of the prediction.

Facing the decarbonization trend in power systems, there appears to be a growing requirement on agile response and delicate supply from electricity suppliers. To accommodate this request, it is of key significance to precisely extrapolate the upcoming power load, which is well acknowledged as VSTLF, i.e., Very Short-Term Power Load Forecasting. As a time series forecasting problem, the primary challenge of VSTLF is how to identify potential factors and their very long-term affecting mechanisms in load demands. With the help of a public dataset, this paper first locates several intensely related attributes based on Pearson’s correlation coefficient and then proposes an adaptive Informer network with the probability sparse attention to model the long-sequence power loads. Additionally, it uses the Shapley Additive Explanations (SHAP) for ablation and interpretation analysis. The experiment results show that the proposed model outperforms several state-of-the-art solutions on several metrics, e.g., 18.39% on RMSE, 21.70% on MAE, 21.24% on MAPE, and 2.11% on R2.

Long sequence time-series forecasting (LSTF) is a significant and challenging task. Many real-world applications require long-term forecasting of time series. In recent years, Transformer-based models have emerged as a promising solution for addressing LSTF tasks. Nevertheless, the model’s performance is constrained by several issues, including the single time scale, the quadratic calculation complexity of the self-attention mechanism, and the high memory occupation. Based on the limitations mentioned above, we propose a novel approach in this paper, namely the multiscale residual sparse attention model RSMformer, built upon the Transformer architecture. Firstly, a residual sparse attention (RSA) mechanism is devised to select dominant queries for computation, utilizing the attention sparsity criterion. This approach effectively reduces the computational complexity to O(LlogL). Secondly, we employ a multiscale forecasting strategy to iteratively refine the accuracy of prediction results at multiple scales by utilizing up-and-down sampling techniques and cross-scale centralization schemes, which effectively capture the temporal dependencies at different time scales. Extensive experiments on six publicly available datasets show that RSMformer performs significantly better than the compared state-of-the-art benchmarks and excels in the LSTF tasks.

Recent studies have shown the potential of transformer-based neural networks in increasing prediction capacity. However, classical transformers present several problems such as computational time complexity and high memory requirements, which make Long Sequence Time-Series Forecasting (LSTF) challenging. The contribution to the prediction of time series of flood events using deep learning techniques is examined, with a particular focus on evaluating the performance of the Informer model (a particular implementation of transformer architecture), which attempts to address the previous issues. The predictive capabilities of the Informer model are explored and compared to statistical methods, stochastic models and traditional deep neural networks. The accuracy, efficiency as well as the limits of the approaches are demonstrated via numerical benchmarks relating to real river streamflow applications. Using daily flow data from the River Test in England as the main case study, we conduct a rigorous evaluation of the Informer efficacy in capturing the complex temporal dependencies inherent in streamflow time series. The analysis is extended to encompass diverse time series datasets from various locations (>100) in the United Kingdom, providing insights into the generalizability of the Informer. The results highlight the superiority of the Informer model over established forecasting methods, especially regarding the LSTF problem. For a forecast horizon of 168 days, the Informer model achieves an NSE of 0.8 and maintains a MAPE below 10%, while the second-best model (LSTM) only achieves −0.63 and 25%, respectively. Furthermore, it is observed that the dependence structure of time series, as expressed by the climacogram, affects the performance of the Informer network.

Transformers have attracted increasing interest in time-series forecasting. However, there are two issues for Multi-Head Self-Attention (MHSA) layers in Multivariate Long Sequence Time-series Forecasting (MLSTF): the massive computation resource consumption and the lack of inductive bias for learning the seasonal and trend pattern of time-series sequences. To address these issues, a systematic method is proposed to replace part of the MHSA layers in Transformers with convolutional layers. Specifically, the self-attention patterns are categorized into four types, i.e., diagonal, vertical, block and heterogeneous patterns. The relationships are explored between convolutional layers and MHSA layers exhibiting different self-attention patterns. Based on which, the evaluation metrics are proposed to decide whether to replace MHSA layers with convolutional layers or not. The experimental results on two representative Transformer-based forecasting models show that our method can achieve competitive results with the original Transformer-based forecasting models and greatly reduce their number of parameters and flops. The performance of models on small data sets has also been greatly improved due to the introduction of convolutional operations. Further, this method is adapted to Transformer-based models for other time series tasks and achieves similar results.

Long sequence time-series forecasting (LSTF) problems, such as weather forecasting, stock market forecasting, and power resource management, are widespread in the real world. The LSTF problem requires a model with high prediction accuracy. Recent studies have shown that the transformer model architecture is the most promising model structure for LSTF problems compared with other model architectures. The transformer model has the property of permutation equivalence, which leads to the importance of sequence position encoding, an essential process in model training. Currently, the continuous dynamics models constructed for position encoding using the neural differential equations (neural ODEs) method can model sequence position information well. However, we have found that there are some limitations when neural ODEs are applied to the LSTF problem, including the time cost problem, the baseline drift problem, and the information loss problem; thus, neural ODEs cannot be directly applied to the LSTF problem. To address this problem, we design a binary position encoding-based regularization model for long sequence time-series prediction, named Seformer, which has the following structure: 1) The binary position encoding mechanism, including intrablock and interblock position encoding. For intrablock position encoding, we design a simple ODE method by discretizing the continuum dynamics model, which reduces the time cost required to compute neural ODEs while maintaining their dynamics properties to the maximum extent. In interblock position encoding, a chunked recursive form is adopted to alleviate the baseline drift problem caused by eigenvalue explosion. 2) Information transfer regularization mechanism: By regularizing the model intermediate hidden variables as well as the encoder-decoder connection variables, we can reduce information loss during the model training process while ensuring the smoothness of the position information. Extensive experimental results obtained on six large-scale datasets show a consistent improvement in our approach over the baselines.