Asset Details
Do you wish to reserve the book?
Fine-tuning protein language models boosts predictions across diverse tasks
Hey, we have placed the reservation for you!
By the way, why not check out events that you can attend while you pick your title.
Oops! Something went wrong.
Looks like we were not able to place the reservation. Kindly try again later.
Are you sure you want to remove the book from the shelf?
Fine-tuning protein language models boosts predictions across diverse tasks
Do you wish to request the book?
Fine-tuning protein language models boosts predictions across diverse tasks
Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy
We have requested the book for you!
Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.
Oops! Something went wrong.
Looks like we were not able to place your request. Kindly try again later.
Journal Article
Fine-tuning protein language models boosts predictions across diverse tasks
2024
Overview
Prediction methods inputting embeddings from protein language models have reached or even surpassed state-of-the-art performance on many protein prediction tasks. In natural language processing fine-tuning large language models has become the de facto standard. In contrast, most protein language model-based protein predictions do not back-propagate to the language model. Here, we compare the fine-tuning of three state-of-the-art models (ESM2, ProtT5, Ankh) on eight different tasks. Two results stand out. Firstly, task-specific supervised fine-tuning almost always improves downstream predictions. Secondly, parameter-efficient fine-tuning can reach similar improvements consuming substantially fewer resources at up to 4.5-fold acceleration of training over fine-tuning full models. Our results suggest to always try fine-tuning, in particular for problems with small datasets, such as for fitness landscape predictions of a single protein. For ease of adaptability, we provide easy-to-use notebooks to fine-tune all models used during this work for per-protein (pooling) and per-residue prediction tasks.
The use of protein language model embeddings has demonstrated state-of-the-art performance in many protein prediction tasks. Here, authors investigate parameter-efficient fine-tuning to further enhance performance and make this technique accessible through their easy-to-use notebooks.
Publisher
Nature Publishing Group UK,Nature Publishing Group,Nature Portfolio
Subject
