Meta ESM-2 Fold - AI faster than Alphafold 2

In this video I go over latest progress from Meta on protein folding, their ESM-2 model used to predict structures of metagenic proteins. This is a big step forward in the field.
In the paper they compare the performance to Alphafold 2 and RoseTTA.
Paper Title: Evolutionary-scale prediction of atomic level protein structure with a language model
Abstract:
Artificial intelligence has the potential to open insight into the structure of proteins at the scale of evolution. It has only recently been possible to extend protein structure prediction to two hundred million cataloged proteins. Characterizing the structures of the exponentially growing billions of protein sequences revealed by large scale gene sequencing experiments would necessitate a breakthrough in the speed of folding. Here we show that direct inference of structure from primary sequence using a large language model enables an order of magnitude speed-up in high resolution structure prediction. Leveraging the insight that language models learn evolutionary patterns across millions of sequences, we train models up to 15B parameters, the largest language model of proteins to date. As the language models are scaled they learn information that enables prediction of the three-dimensional structure of a protein at the resolution of individual atoms. This results in prediction that is up to 60x faster than state-of-the-art while maintaining resolution and accuracy. Building on this, we present the ESM Metagenomic Atlas. This is the first large-scale structural characterization of metagenomic proteins, with more than 617 million structures. The atlas reveals more than 225 million high confidence predictions, including millions whose structures are novel in comparison with experimentally determined structures, giving an unprecedented view into the vast breadth and diversity of the structures of some of the least understood proteins on earth.
Timestamps:
0:00 Intro
1:18 ESMfold Intro
1:45 Metagenomics
2:11 Protein Folding Background
3:07 Model Architecture
4:51 Results
6:42 Results vs. Aplhafold 2
7:47 Why is Meta doing this?
8:31 Practical Uses
References:
Original Paper: www.biorxiv.org/content/10.11...
Meta Blog: / protein-folding-esmfol...
Nature Journal Highlight: www.nature.com/articles/d4158...
MSA Transformer:
www.biorxiv.org/content/10.11...
Videos:
Deepmind AlphaFold:
• AlphaFold: The making ...
Images:
en.wikipedia.org/wiki/Amino_a...
en.wikipedia.org/wiki/Hemoglo...

[@WitsMethod]

Great 👍 love 💕 from Bangladesh 🇧🇩

[@aogozen]

Great job, Matej! You were always good at explaining things!

[@AIMatej]

Thanks Omer! I hope that you are doing great!

[@BrienDunn]

Excellent. Thanks for making this informative video. You should have 100x the subscribers.

[@AIMatej]

So nice of you to say!

[@AbhishekS-cv3cr]

Excellent Content!

[@filipsand3290]

Great content, keep up the good work.

[@AIMatej]

Appreciate it!

[@jouhanehajali3564]

شكرا لكم استاذنا. من الرائع جدا اكتشاف كيفية طيّ البروتين. Thank you sir so much.

[@AIMatej]

I am glad that you have enjoyed the video:)

[@studyforsuccess3899]

Thank you so much ☺️

[@AIMatej]

You’re welcome 😊

[@almor2445]

Does this resolve the quality issues mentioned on the wiki page of protein folding that claims 1/3 of the data produced by AlphaFold was unusable?

[@Jaeoh.woof765]

what do you mean by "unusable"? low quality?

[@samuelsaldana1575]

Hi, appear to the content, ColabFold and another variants of the same AlphaFold2: OmegaFold2 (is superiority in short sequences). However, ESM Fold is the fastest method for sequences with length 50 and 100. But, ESMFold it is not as accurate as Omega Fold or ColabFold. Both (Colab & Omega), are accurate methods than ESMFold. Still missing bring OpenFold.

[@eaudesolero5631]

ok so what is the difference between these *Fold systems and the LLM's that have gotten so much attention in the last few weeks? are they different then the systems that can produce the most efficient structural designs for things like vehicles or furniture frames? what is Alpha Tensor? what other types of systems are there?

[@AIMatej]

All of the models mentioned are based on the transformer architecture developed by google in 2017. But Alpha fold, ESM-2 fold and alpha tensor are very customized systems capable of amazing things, but they don't have a customer interface and are used for very specific purposes. The reason why ChatGPT/GPT-4 are getting so much press time is because normal people can try them out and the output is useful. So far I am not aware of anything that GPT-4 did that surpassed quality of human performance (but it is much faster). Alphafold and alphatensor did something that humans just could not do.

[@squamish4244]

@@AIMatej These have a far more serious near-term application i.e. biology, but not a lot of people are even aware of them. The implications for basically every disease in existence and aging are immense. This year, a potential liver cancer drug was discovered in 30 days by AlphaFold.

[@paulusbrent9987]

Your "honestly I'm not entirely sure" at 7:53 speaks volumes... 😀 That's all fascinating and fine. But it remains an estimating tool that will always need empiric verification to check if the prediction is correct. And, at the end of the day, I doubt that, apart from theoretical new insights (yes, evolutionary biology might take great advantage of it). I don't think it will lead us to real-life practical applications such as curing diseases. Because we are light years away from filling the gap between protein structure and organic functions. We have no idea why a specific architecture leads to specific functionality. In fact, even once we will know all the protein structures, that will not automatically tell us how to design from it new drugs. Protein structure by itself will not be more informative to design new drugs as the mapping of the genome was for designing drugs against genetic diseases. As usual, again and again, it turns out that the map is not the territory. Anyway, thank you for updating us.

[@AIMatej]

I think you misunderstood what I meant by that comment: I don't understand why Meta is working on this, they are a social network, VR, metaverse company. Meta's goal is to make money, I am not sure how this helps them increase revenue. The work itself is extremely useful and structures of proteins are extremely useful or many ways. And some proteins are very hard/impossible to determine experimentally.

[@paulusbrent9987]

@@AIMatej Ok, can you make some examples of how the knowledge of the structure of proteins became useful?

[@AIMatej]

@@paulusbrent9987 www.nature.com/articles/s42003-021-02261-4

[@paulusbrent9987]

@@AIMatej As far as I understand it, this explains in hindsight why an antibiotic works due to molecular structure. It was not the knowledge of this structure that led to its creation. I'm wondering if there is a case where the knowledge of a protein structure led to a new drug? After all this is what Alfafold is supposed to do in practical applications.

[@calebcooper2333]

@@paulusbrent9987 Yes, pharmaceutical companies use protein structure all the time to design drugs. In a lot of cases, they produce drugs that target or attach to very specific proteins. You can simply login to your college library or use scihub to discover articles in which protein structure is targeted for drug binding, amongst many other use cases. Even before AI, we discovered protein structures and developed drugs and therapeutics to target those aspects. Go do some research and get off of KZbin.

[@itsjaysenofficial]

this is also is why people worry about vaccines.