Explore the vast range of titles available.

Negative sampling is a common approach for making the training of deep models in classification problems with very large output spaces, known as extreme multilabel classification (XMC) problems, tractable. Negative sampling methods aim to find per instance negative labels with higher scores, known as hard negatives, and limit the computations of the negative part of the loss to these labels. Two well-known methods for negative sampling in XMC models are meta-classifier-based and Maximum Inner product Search (MIPS)-based adaptive methods. Owing to their good prediction performance, methods which employ a meta classifier are more common in contemporary XMC research. On the flip side, they need to train and store the meta classifier (apart from the extreme classifier), which can involve millions of additional parameters. In this paper, we focus on the MIPS-based methods for negative sampling. We highlight two issues which may prevent deep models trained by these methods to undergo stable training. First, we argue that using hard negatives excessively from the beginning of training leads to unstable gradient. Second, we show that when all the negative labels in a MIPS-based method are restricted to only those determined by MIPS, training is sensitive to the length of intervals for pre-processing the weights in the MIPS method. To mitigate the aforementioned issues, we propose to limit the labels selected by MIPS to only a few and sample the rest of the needed labels from a uniform distribution. We show that our proposed MIPS-based negative sampling can reach the performance of LightXML, a transformer-based model trained by a meta classifier, while there is no need to train and store any additional classifier. The code for our experiments is available at https://github.com/xmc-aalto/mips-negative-sampling .

News recommendation is an intelligent technology that aims to provide users with matching news content based on their preferences and interests. Nevertheless, current methodologies exhibit significant limitations. Traditional models often rely on simple random negative sampling for training, an approach that insufficiently captures the patterns and preferences of users’ clicking behavior, thereby undermining the model’s effectiveness. Furthermore, these systems often face challenges in insufficient modeling due to the limited nature of user interactions. Considering these challenges, this paper presents a contrastive news recommendation framework based on curriculum learning (CNRCL). Specifically, we relate the negative sampling process to users’ interests and employ curriculum learning to guide the negative sampling procedure. To address the issue of insufficient user interest modeling, we propose to use contrastive learning to bring the user closer to news that is similar to the candidate news, thus enhancing the model’s accuracy in predicting user interests, and compensating for limited click behavior. Extensive experimental results on the MIND dataset verify the effectiveness of the model and improve the performance of news recommendation. Our code can be obtained from https://github.com/IIP-Lab-2024/CNRCL.

Contrastive learning has emerged as an essential approach in self-supervised visual representation learning. Its main goal is to maximize the similarities between augmented versions of the same image (positive pairs), while minimizing the similarities between different images (negative pairs). Recent studies have demonstrated that harder negative samples, i.e., those that are more challenging to differentiate from the anchor sample perform a more crucial function in contrastive learning. However, many existing contrastive learning methods ignore the role of hard negative samples. In order to provide harder negative samples for the network model more efficiently. This paper proposes a novel feature-level sample sampling method, namely sampling synthetic hard negative samples for contrastive learning (SSCL). Specifically, we generate more and harder negative samples by mixing them through linear combination and ensure their reliability by debiasing. Finally, we execute weighted sampling of these negative samples. Compared to state-of-the-art methods, our method can provide more high-quality negative samples. Experiments show that SSCL improves the classification performance on different image datasets and can be readily integrated into existing methods.

Recently, knowledge graphs (KGs) have been shown to benefit many machine learning applications in multiple domains (e.g. self-driving, agriculture, bio-medicine, recommender systems, etc.). However, KGs suffer from incompleteness, which motivates the task of KG completion which consists of inferring new (unobserved) links between existing entities based on observed links. This task is achieved using either a probabilistic, rule-based, or embedding-based approach. The latter has been shown to consistently outperform the former approaches. It however relies on negative sampling, which supposes that every observed link is “true” and that every unobserved link is “false”. Negative sampling increases the computation complexity of the learning process and introduces noise in the learning. We propose NSF-KGE, a framework for KG embedding that does not require negative sampling, yet achieves performance comparable to that of the negative sampling-based approach. NSF-KGE employs objectives from the non-contrastive self-supervised literature to learn representations that are invariant to relation transformations (e.g. translation, scaling, rotation etc) while avoiding representation collapse.

Background Imbalanced datasets are commonly encountered in bioinformatics classification problems, that is, the number of negative samples is much larger than that of positive samples. Particularly, the data imbalance phenomena will make us underestimate the performance of the minority class of positive samples. Therefore, how to balance the bioinformatic data becomes a very challenging and difficult problem. Results In this study, we propose a new data sampling approach, called pseudo-negative sampling, which can be effectively applied to handle the case that: negative samples greatly dominate positive samples. Specifically, we design a supervised learning method based on a max-relevance min-redundancy criterion beyond Pearson correlation coefficient (MMPCC), which is used to choose pseudo-negative samples from the negative samples and view them as positive samples. In addition, MMPCC uses an incremental searching technique to select optimal pseudo-negative samples to reduce the computation cost. Consequently, the discovered pseudo-negative samples have strong relevance to positive samples and less redundancy to negative ones. Conclusions To validate the performance of our method, we conduct experiments base on four UCI datasets and three real bioinformatics datasets. According to the experimental results, we clearly observe the performance of MMPCC is better than other sampling methods in terms of Sensitivity, Specificity, Accuracy and the Mathew’s Correlation Coefficient. This reveals that the pseudo-negative samples are particularly helpful to solve the imbalance dataset problem. Moreover, the gain of Sensitivity from the minority samples with pseudo-negative samples grows with the improvement of prediction accuracy on all dataset.

Sampling high-quality negative samples and training together with positive samples can help improve the performance and generalization ability of the recommendation model. However, traditional negative sampling methods, such as random sampling or heuristic rules, often fail to adequately capture negative samples that reflect the user’s true taste. To provide interpretability and diversity of negative samples, we propose a collaborative knowledge graph-based reinforcement negative sampling recommendation model called KGRec-RNS that formalizes the problem of finding negative signals into a Markov decision process (MDP). Firstly, user interaction data and external knowledge are integrated into a collaborative Bipartite-Knowledge graph (BKG) as MDP context information environment. Then, an Actor based sampler including graph learning module, neighbor attention module and neighbor pruning module is designed. The graph learning module utilizes graph convolutional neural network (GCN) to extract high-order information of each node, the neighbor attention module is adopted to distinguish the influence of different nodes, and the user conditional action pruning strategy is integrated. Thus, negative samples with interpretability can be screened out. Finally, a Critic based recommender was designed to evaluate the current state and the action reward to guide the policy update of the Actor network, thereby matching high-quality negative samples with positive samples. The experimental results on three real datasets demonstrate that our KGRec-RNS has significant advantages in providing more accurate and diverse recommendations.

Background Understanding protein-molecular interaction is crucial for unraveling the mechanisms underlying diverse biological processes. Machine learning (ML) techniques have been extensively employed in predicting these interactions and have garnered substantial research focus. Previous studies have predominantly centered on improving model performance through novel and efficient ML approaches, often resulting in overoptimistic predictive estimates. However, these advancements frequently neglect the inherent biases stemming from network properties, particularly in biological contexts. Results In this study, we examined the biases inherent in ML models during the learning and prediction of protein-molecular interactions, particularly those arising from the scale-free property of biological networks—a characteristic where in a few nodes have many connections while most have very few. Our comprehensive analysis across diverse tasks, datasets, and ML methods provides compelling evidence of these biases. We discovered that the training and evaluation of ML models are profoundly influenced by network topology, potentially distorting model performance assessments. To mitigate this issue, we propose the degree distribution balanced (DDB) sampling strategy, a straightforward yet potent approach that alleviates biases stemming from network properties. This method further underscores the limitations of certain ML models in learning protein-molecular interactions solely from intrinsic molecular features. Conclusions Our findings present a novel perspective for assessing the performance of ML models in inferring protein-molecular interactions with greater fairness. By addressing biases introduced by network properties, the DDB sampling approach provides a more balanced and precise assessment of model capabilities. These insights hold the potential to bolster the reliability of ML models in bioinformatics, fostering a more stringent evaluation framework for predicting protein-molecular interactions.

Network embedding can effectively dig out potentially useful information and discover the relationships and rules which exist in the data, that has attracted increasing attention in many real-world applications. The goal of network embedding is to map high-dimensional and sparse networks into low-dimensional and dense vector representations. In this paper, we propose a network embedding method based on high-degree penalty and adaptive negative sampling (NEPS). First, we analyze the problem of imbalanced node training in random walk and propose an indicator base on high-degree penalty, which can control the random walk and avoid over-sampling high-degree neighbor node. Then, we propose a two-stage adaptive negative sampling strategy, which can dynamically obtain negative samples suitable for the current training according to the training stage to improve training effect. By comparing with seven well-known network embedding algorithms on eight real-world data sets, experiments show that the NEPS has good performance in node classification, network reconstruction and link prediction. The code is available at: https://github.com/Andrewsama/NEPS-master.

Vanilla Transformers focus on semantic relevance between mid‐ to high‐level features and are not good at extracting smoke features, as they overlook subtle changes in low‐level features like color, transparency, and texture, which are essential for smoke recognition. To address this, we propose the cross contrast patch embedding (CCPE) module based on the Swin Transformer. This module leverages multiscale spatial contrast information in both vertical and horizontal directions to enhance the network’s discrimination of underlying details. By combining cross contrast with the transformer, we exploit the advantages of the transformer in the global receptive field and context modeling while compensating for its inability to capture very low‐level details, resulting in a more powerful backbone network tailored for smoke recognition tasks. In addition, we introduce the separable negative sampling mechanism (SNSM) to address supervision signal confusion during training and release the SKLFS‐WildFire test dataset, the largest real‐world wildfire test set to date, for systematic evaluation. Extensive testing and evaluation on the benchmark dataset FIgLib and the SKLFS‐WildFire test dataset show significant performance improvements of the proposed method over baseline detection models.

Speaker recognition is a technology that identifies the speaker in an input utterance by extracting speaker-distinguishable features from the speech signal. Speaker recognition is used for system security and authentication; therefore, it is crucial to extract unique features of the speaker to achieve high recognition rates. Representative methods for extracting these features include a classification approach, or utilizing contrastive learning to learn the speaker relationship between representations and then using embeddings extracted from a specific layer of the model. This paper introduces a framework for developing robust speaker recognition models through contrastive learning. This approach aims to minimize the similarity to hard negative samples—those that are genuine negatives, but have extremely similar features to the positives, leading to potential mistaken. Specifically, our proposed method trains the model by estimating hard negative samples within a mini-batch during contrastive learning, and then utilizes a cross-attention mechanism to determine speaker agreement for pairs of utterances. To demonstrate the effectiveness of our proposed method, we compared the performance of a deep learning model trained with a conventional loss function utilized in speaker recognition with that of a deep learning model trained using our proposed method, as measured by the equal error rate (EER), an objective performance metric. Our results indicate that when trained with the voxceleb2 dataset, the proposed method achieved an EER of 0.98% on the voxceleb1-E dataset and 1.84% on the voxceleb1-H dataset.