As the capabilities of generative AI models have grown, you've probably seen how they will transform easy text prompts into hyper-realistic images and even enhanced video clips.
More recently, generative AI has shown that it might probably help chemists and biologists explore static molecules like proteins and DNA. Models like AlphaFold can predict molecular structures to speed up drug discovery, and the MIT-backed “RF transmission“For example, it might probably help develop latest proteins. One challenge, nevertheless, is that molecules are consistently moving and moving forwards and backwards, which is essential when designing latest proteins and medicines. Simulating these movements on a pc using physics – a way often known as molecular dynamics – may be very expensive, requiring billions of time steps on supercomputers.
To simulate these behaviors more efficiently, researchers on the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Department of Mathematics have developed a generative model that learns from previous data. The team's system, called MDGen, can take a picture of a 3D molecule and, like a video, simulate what is going to occur next, join separate still images, and even fill in missing images. By pressing play on molecules, the tool could potentially help chemists design latest molecules and study exactly how well their drug prototypes for cancer and other diseases would interact with the molecular structure it is meant to influence.
Co-lead writer Bowen Jing SM '22 says MDGen is an initial proof of concept, however it suggests the start of an exciting latest direction of research. “Early on, generative AI models produced quite easy videos, similar to an individual blinking or a dog wagging its tail,” says Jing, a doctoral student at CSAIL. “A couple of years later we have now amazing models like Sora or Veo that may be useful in lots of interesting ways. We hope to convey an analogous vision for the molecular world, where dynamic trajectories are the videos. For example, you’ll be able to give the model the primary and tenth frames and it animates what’s in between, or it might probably remove noise from a molecular video and guess what was hidden.”
The researchers say MDGen represents a paradigm shift from previous comparable work with generative AI in a way that allows much broader use cases. Previous approaches were “autoregressive,” meaning they relied on the previous still image to create the following one, starting with the very first frame to create a video sequence. In contrast, MDGen generates the frames in parallel with diffusion. This signifies that MDGen may be used, for instance, to affix frames on the endpoints or to “upsample” a low framerate trajectory along with pressing play on the primary frame.
This work was presented in a paper presented on the Conference on Neural Information Processing Systems (NeurIPS) last December. Last summer it was recognized for its potential business impact on the ML4LMS workshop on the International Conference on Machine Learning.
Some small advances for molecular dynamics
In experiments, Jing and his colleagues found that MDGen's simulations were just like running physical simulations directly, but produced trajectories 10 to 100 times faster.
The team first tested their model's ability to take a 3D frame of a molecule and generate the following 100 nanoseconds. Their system pieced together consecutive 10-nanosecond blocks to permit these generations to succeed in that duration. The team found that MDGen could match the accuracy of a baseline model, completing the video generation process in a couple of minute – a fraction of the three hours it took the baseline model to simulate the identical dynamics.
When MDGen received the primary and last frames of a one-nanosecond sequence, it also modeled the steps in between. The researchers' system showed a level of realism across over 100,000 different predictions: It simulated more likely molecular trajectories than its baselines on clips shorter than 100 nanoseconds. In these tests, MDGen also demonstrated the flexibility to generalize peptides that it had not seen before.
MDGen's capabilities also include simulating frames inside frames and upsampling the steps between each nanosecond to raised capture faster molecular phenomena. It may even “color” structures of molecules, restoring distant details about them. These features could eventually be utilized by researchers to design proteins based on a specification of how different parts of the molecule should move.
Playing around with protein dynamics
Jing and co-lead writer Hannes Stärk say MDGen is an early sign of progress in generating molecular dynamics more efficiently. Still, they lack the info to make these models immediately useful for developing drugs or molecules that trigger the movements that chemists need to see in a goal structure.
Researchers need to scale MDGen from modeling molecules to predicting how proteins change over time. “We are currently using toy systems,” says Stärk, also a doctoral student at CSAIL. “To improve MDGen’s predictive capabilities for modeling proteins, we’d like to construct on the present architecture and available data. We don’t yet have a YouTube-scale repository for these kind of simulations, so we hope to develop a separate machine learning method that may speed up the info collection process for our model.”
Currently, MDGen represents an encouraging path forward in modeling molecular changes which are invisible to the naked eye. Chemists could also use these simulations to delve deeper into the behavior of drug prototypes for diseases similar to cancer or tuberculosis.
“Machine learning methods that learn from physical simulations represent a burgeoning latest frontier in AI for science,” says MIT Simons Professor of Mathematics Bonnie Berger, CSAIL principal investigator and senior writer of the paper. “MDGen is a flexible, versatile modeling framework that bridges these two areas, and we’re very excited to share our early models on this direction.”
“Sampling realistic transition paths between molecular states is a significant challenge,” says co-author and senior writer Tommi Jaakkola, Thomas Siebel Professor of Electrical Engineering and Computer Science at MIT and Institute for Data, Systems and Society, and CSAIL principal investigator. “This early work shows how we would begin to deal with such challenges by moving generative modeling to full simulation runs.”
Researchers across the bioinformatics field have praised this method for its ability to simulate molecular transformations. “MDGen models molecular dynamics simulations as a shared distribution of structural embeddings and captures molecular movements between discrete time steps,” says Simon Olsson, an associate professor at Chalmers University of Technology, who was not involved within the research. “MDGen leverages a masked learning objective and enables modern use cases similar to sampling transition paths and drawing analogies for inpainting trajectories connecting metastable phases.”
The researchers' work on MDGen was supported partially by the National Institute of General Medical Sciences, the U.S. Department of Energy, the National Science Foundation, the Machine Learning for Pharmaceutical Discovery and Synthesis Consortium, and the Abdul Latif Jameel Clinic for Machine Learning in Health, Defense Threat Reduction Agency and the Defense Advanced Research Projects Agency.
