Journal Article

Amino Acid Encoding for Deep Learning Applications

Bromberg, Yana, Lenz, Tobias, Wendorff, Mareike, ElAbd, Hesham, Hoarfrost, Adrienne

2020

Overview

Background: The number of applications of deep learning algorithms in bioinformatics is increasing as they usually achieve superior performance over classical approaches, especially, when bigger training datasets are available. In deep learning applications, discrete data, e.g. words or n-grams in language, or amino acids or nucleotides in bioinformatics, are generally represented as a continuous vector through an embedding matrix. Recently, learning this embedding matrix directly from the data as part of the continuous iteration of the model to optimize the target prediction – a process called ‘end-to-end learning’ – has led to state-of-the-art results in many fields. Although usage of embeddings is well described in the bioinformatics literature, the potential of end-to-end learning for single amino acids, as compared to more classical manually-curated encoding strategies, has not been systematically addressed. To this end, we compared classical encoding matrices, namely one-hot, VHSE8 and BLOSUM62, to end-to-end learning of amino acid embeddings for two different prediction tasks using three widely used architectures, namely recurrent neural networks (RNN), convolutional neural networks (CNN), and the hybrid CNN-RNN. Results: By using different deep learning architectures, we show that end-to-end learning is on par with classical encodings for embeddings of the same dimension even when limited training data is available, and might allow for a reduction in the embedding dimension without performance loss, which is critical when deploying the models to devices with limited computational capacities. We found that the embedding dimension is a major factor in controlling the model performance. Surprisingly, we observed that deep learning models are capable of learning from random vectors of appropriate dimension. Conclusion: Our study shows that end-to-end learning is a flexible and powerful method for amino acid encoding. Further, due to the flexibility of deep learning systems, amino acid encoding schemes should be benchmarked against random vectors of the same dimension to disentangle the information content provided by the encoding scheme from the distinguishability effect provided by the scheme.

• Publisher: BMC,BioMed Central,BioMed Central Ltd,Springer Nature B.V
• Subject: Algorithms / Amino acid encoding / Amino acids / Amino Acids - metabolism / Amino acids embedding / Artificial neural networks / Bioinformatics / Biomedical and Life Sciences / Cable television broadcasting industry / Computational Biology - methods / Computational Biology/Bioinformatics / Computer Appl. in Life Sciences / Computer applications / Convoluted-neural network (CNN) / Cybernetics, Artificial Intelligence and Robotics / Data mining / Deep learning / Deep Learning - standards / Embedding / HLA-II peptide interaction / Humans / Iterative methods / Learning algorithms / Life Sciences / Machine learning / Machine Learning and Artificial Intelligence in Bioinformatics / Mathematical analysis / Matrix algebra / Matrix methods / Methodology / Methodology Article / Microarrays / Neural networks / Nucleotides / Observational learning / Peptides / Protein-protein interaction (PPI) / Protein-protein interactions / Proteins / Recurrent neural networks / Technology application / Training