Explore the vast range of titles available.

Pre-trained Transformer-based models have achieved state-of-the-art performance for various Natural Language Processing (NLP) tasks. However, these models often have billions of parameters, and thus are too resource- hungry and computation-intensive to suit low- capability devices or applications with strict latency requirements. One potential remedy for this is model compression, which has attracted considerable research attention. Here, we summarize the research in compressing Transformers, focusing on the especially popular BERT model. In particular, we survey the state of the art in compression for BERT, we clarify the current best practices for compressing large-scale Transformer models, and we provide insights into the workings of various methods. Our categorization and analysis also shed light on promising future research directions for achieving lightweight, accurate, and generic NLP models.

One of the challenges in a viral pandemic is the emergence of novel variants with different phenotypical characteristics. An ability to forecast future viral individuals at the sequence level enables advance preparation by characterizing the sequences and closing vulnerabilities in current preventative and therapeutic methods. In this article, we explore, in the context of a viral pandemic, the problem of generating complete instances of undiscovered viral protein sequences, which have a high likelihood of being discovered in the future using protein language models. Current approaches to training these models fit model parameters to a known sequence set, which does not suit pandemic forecasting as future sequences differ from known sequences in some respects. To address this, we develop a novel method, called PandoGen, to train protein language models towards the pandemic protein forecasting task. PandoGen combines techniques such as synthetic data generation, conditional sequence generation, and reward-based learning, enabling the model to forecast future sequences, with a high propensity to spread. Applying our method to modeling the SARS-CoV-2 Spike protein sequence, we find empirically that our model forecasts twice as many novel sequences with five times the case counts compared to a model that is 30× larger. Our method forecasts unseen lineages months in advance, whereas models 4× and 30× larger forecast almost no new lineages. When trained on data available up to a month before the onset of important Variants of Concern, our method consistently forecasts sequences belonging to those variants within tight sequence budgets.

Background Acute lung injury (ALI) is a common and serious complication of sepsis with high mortality. Ferroptosis, categorized as programmed cell death, contributes to the development of lung injury. Protectin conjugates in tissue regeneration 1 (PCTR1) is an endogenous lipid mediator that exerts protective effects against multiorgan injury. However, the role of PCTR1 in the ferroptosis of sepsis-related ALI remains unknown. Methods A pulmonary epithelial cell line and a mouse model of ALI stimulated with lipopolysaccharide (LPS) were established in vitro and in vivo. Ferroptosis biomarkers, including ferrous (Fe 2+ ), glutathione (GSH), malondialdehyde (MDA) and 4-Hydroxynonenal (4-HNE), were assessed by relevant assay kits. Glutathione peroxidase 4 (GPX4) and prostaglandin-endoperoxide synthase 2 (PTGS2) protein levels were determined by western blotting. Lipid peroxides were examined by fluorescence microscopy and flow cytometry. Cell viability was determined by a CCK-8 assay kit. The ultrastructure of mitochondria was observed with transmission electron microscopy. Morphology and inflammatory cytokine levels predicted the severity of lung injury. Afterward, related inhibitors were used to explore the potential mechanism by which PCTR1 regulates ferroptosis. Results PCTR1 treatment protected mice from LPS-induced lung injury, which was consistent with the effect of the ferroptosis inhibitor ferrostatin-1. PCTR1 treatment decreased Fe 2+ , PTGS2 and lipid reactive oxygen species (ROS) contents, increased GSH and GPX4 levels and ameliorated mitochondrial ultrastructural injury. Administration of LPS or the ferroptosis agonist RSL3 resulted in reduced cell viability, which was rescued by PCTR1. Mechanistically, inhibition of the PCTR1 receptor lipoxin A4 (ALX), protein kinase A (PKA) and transcription factor cAMP-response element binding protein (CREB) partly decreased PCTR1 upregulated GPX4 expression and a CREB inhibitor blocked the effects ofPCTR1 on ferroptosis inhibition and lung protection. Conclusion This study suggests that PCTR1 suppresses LPS-induced ferroptosis via the ALX/PKA/CREB signaling pathway, which may offer promising therapeutic prospects in sepsis-related ALI.

Background Modern Next Generation- and Third Generation- Sequencing methods such as Illumina and PacBio Circular Consensus Sequencing platforms provide accurate sequencing data. Parallel developments in Deep Learning have enabled the application of Deep Neural Networks to variant calling, surpassing the accuracy of classical approaches in many settings. DeepVariant, arguably the most popular among such methods, transforms the problem of variant calling into one of image recognition where a Deep Neural Network analyzes sequencing data that is formatted as images, achieving high accuracy. In this paper, we explore an alternative approach to designing Deep Neural Networks for variant calling, where we use meticulously designed Deep Neural Network architectures and customized variant inference functions that account for the underlying nature of sequencing data instead of converting the problem to one of image recognition. Results Results from 27 whole-genome variant calling experiments spanning Illumina, PacBio and hybrid Illumina-PacBio settings suggest that our method allows vastly smaller Deep Neural Networks to outperform the Inception-v3 architecture used in DeepVariant for indel and substitution-type variant calls. For example, our method reduces the number of indel call errors by up to 18%, 55% and 65% for Illumina, PacBio and hybrid Illumina-PacBio variant calling respectively, compared to a similarly trained DeepVariant pipeline. In these cases, our models are between 7 and 14 times smaller. Conclusions We believe that the improved accuracy and problem-specific customization of our models will enable more accurate pipelines and further method development in the field. HELLO is available at https://github.com/anands-repo/hello

China’s BeiDou Ground-based Augmentation System (GBAS) improves the accuracy of BeiDou satellites navigation system to the sub-meter level. GBAS is widely used in the navigation fields, such as civil vehicles, ships, drones, which produces significant economic benefits and lays the foundation for application of high dynamic missile. According to the application requirements of guided rocket, this paper designed a high dynamic real time differential (RTD) based on GBAS, and proposed the composition, principle, working flow, missile equipment devices and ground terminal, etc. Through the design analysis, it is hoped that this paper can provide reference for high dynamic application of GBAS.

Fermentation ability and alcohol production ability are important quality indicators of Chinese liquor Daqu, reflecting microbial growth and metabolic capacity and ethanol production capacity of Daqu microbiota, respectively. However, information on the microbial community related to the fermentation ability and alcohol production ability is unclear. In this study, fermentation functional microbiota (FFM) and alcohol functional microbiota (AFM) were obtained by correlating fermentation ability and alcohol production ability with Daqu microbiota. FFM and AFM consisted of 50 and 49 genera, respectively, which were basically the same at the phylum level but differed at the genus level. Correlation analysis showed that FFM and AFM were mainly affected by moisture, acidity, and humidity in the early stage of Daqu fermentation, and oxygen content was a critical factor for microbial succession in the middle stage of fermentation. FFM and AFM had commensal or synergistic interactions with multiple microbes. Function predictions indicated that fermentation functional bacterial microbiota was active in product synthesis and transport-related metabolic functions, and alcohol functional bacterial microbiota was very active in raw material utilization and its own metabolic synthesis. This study reveals the structural characteristics and formation mechanism of FFM and AFM, which is important for control of Daqu quality.

Analyzing the dynamics of parking traffic can better represent the real dynamic states of road networks, thereby allowing for a deeper analysis of the parking system’s impact. This paper comparatively investigates the impact of parking policies on two traffic networks with different infrastructure, socio-economic, and policy characteristics. Parking space, average parking duration, and parking fee policies were analyzed as a function of cruising distances and cruising time with indirect effects on traffic emissions. Empirically, the system dynamics model application is tested and validated with the macroscopic data from two central business districts (CBDs) in Shanghai (Xujiahui area) and Zurich (Bahnhofstrasse area). Results showed Bahnhofstrasse CBD is more sensitive against the policy shifts with relatively higher elasticity and indicated greater responsiveness in aggregating traffic emissions when compared with Xujiahui CBD. The findings of this study may provide an overall framework to empirically assess the performance of different traffic conditions and strategies on urban parking systems.

With the rapid iteration of computer information technology, the application of computer-based configuration software in building electrical equipment can further improve the functionality of the building itself, meet people's needs for building residence, office and other activities, improve people's living and use experience and feelings, and help to achieve energy conservation and emission reduction, and reduce the use cost of buildings, therefore, it has important research value. Based on this, this paper first analyses the characteristics and application process of the configuration software of the computer Internet of things system, and then gives the construction and application strategy of the configuration software of the computer Internet of things for building electrical equipment.

The integrated development of offshore wind power and marine aquaculture represents a promising approach to the sustainable utilization of ocean resources. The present study investigated the hydrodynamic response of an innovative combination of a wind farm monopile and pontoon raft aquaculture facilities (PRAFs). Physical water tank experiments were conducted on PRAFs deployed around a wind farm monopile using the following configurations: single- and three-row arrangements of PRAFs with and without a monopile. The interaction between the aquaculture structure and the wind farm monopile was examined, with a particular focus on the mooring line tensions and bridle line tensions under different wave conditions. Utilizing the wind farm monopile foundation as an anchor, the mooring line tension was reduced significantly by 16–66% in the single-row PRAF. The multi-row PRAF arrangement experienced lower mooring line tension in comparison with the single-row PRAF arrangement, with the highest reduction of 73%. However, for the bridle line tension, the upstream component was enhanced, while the downstream one was weakened with a monopile, and they both decreased in the multi-row arrangement. Finally, we developed numerical models based on flume tank tests that examined the interactions between the monopile and PRAFs, including configurations of a single monopile, along with single- and three-row arrangements of PRAFs. The numerical simulation results confirmed that the monopile had a dampening effect on the wave propagation of 5% to 20%, and the impact of the pontoons on the monopile was negligible, implying that the integration of aquaculture facilities around wind farm infrastructure may not significantly alter the hydrodynamic loads experienced by the monopile.

FPGAs are an attractive platform for applications with high computation demand and low energy consumption requirements. However, design effort for FPGA implementations remains high—often an order of magnitude larger than design effort using high-level languages. Instead of this time-consuming process, high-level synthesis (HLS) tools generate hardware implementations from algorithm descriptions in languages such as C/C++ and SystemC. Such tools reduce design effort: high-level descriptions are more compact and less error prone. HLS tools promise hardware development abstracted from software designer knowledge of the implementation platform. In this paper, we present an unbiased study of the performance, usability and productivity of HLS using AutoPilot (a state-of-the-art HLS tool). In particular, we first evaluate AutoPilot using the popular embedded benchmark kernels. Then, to evaluate the suitability of HLS on real-world applications, we perform a case study of stereo matching, an active area of computer vision research that uses techniques also common for image denoising, image retrieval, feature matching, and face recognition. Based on our study, we provide insights on current limitations of mapping general-purpose software to hardware using HLS and some future directions for HLS tool development. We also offer several guidelines for hardware-friendly software design. For popular embedded benchmark kernels, the designs produced by HLS achieve 4X to 126X speedup over the software version. The stereo matching algorithms achieve between 3.5X and 67.9X speedup over software (but still less than manual RTL design) with a fivefold reduction in design effort versus manual RTL design.