Target specific peptide design using latent space approximate trajectory collector

doi:10.21203/rs.3.pex-2185/v1

Method Article

Target specific peptide design using latent space approximate trajectory collector

https://doi.org/10.21203/rs.3.pex-2185/v1

This work is licensed under a CC BY 4.0 License

This protocol has been posted on Protocol Exchange, an open repository of community-contributed protocols sponsored by Nature Portfolio. These protocols are posted directly on the Protocol Exchange by authors and are made freely available to the scientific community for use and comment.

Version 1

posted

You are reading this latest protocol version

Despite the prevalence and many successes of deep learning applications in de novo molecular design, the problem of peptide generation targeting specific proteins remains unsolved. A main barrier for this is the scarcity of the high-quality training data. To tackle the issue, we propose a novel machine learning based peptide design architecture, called Latent Space Approximate Trajectory Collector (LSATC). It consists of a series of samplers on an optimization trajectory on a highly non-convex energy landscape that approximates the distributions of peptides with desired properties in a latent space. The process involves little human intervention and can be implemented in an end-to-end manner. We demonstrate the model by the design of peptide extensions targeting β-catenin, a key nuclear effector protein involved in canonical Wnt signalling. When compared with a random sampler, LSATC can sample peptides with 36% lower mean binding scores in a 16 times smaller interquartile range (IQR) and 284% less mean hydrophobicity with a 1.4 times smaller IQR. LSATC also largely outperforms other common generative models.Finally, we utilize a clustering algorithm to select 4 peptides from the100 LSATC designed peptides for experimental validation. The resultconfirms that all the four peptides extended by LSATC show improved β-catenin binding by at least 20.0%, and two of the peptides show a 3 fold increase in binding affinity as compared to the base peptide.

Automated protein specific peptide design

Machine learning

Evolutionary optimization

Therapeutic peptides are a class of pharmaceutical agents that are distinctfrom small molecule drugs due to their unique biochemical and therapeuticcharacteristics. In recent years, many peptide drugs have been found to havesuperior potency and safety profiles than small molecule drugs [1]. Peptidescan disrupt unwanted protein-protein interactions (PPI) that have often beenimplicated to play a role in cancer development and progression. One suchexample is the deregulation of Wnt/beta-catenin/T-cell factor (Tcf) signalingcommon in many human cancers. Thus, the development of peptide-basedPPI inhibitors has become one of the most topical directions in cancer drugresearch.

Structure-based design of peptide inhibitors for a specific protein tar-get has long been an empirical task. Traditionally, peptide design has beenfocused on sequence perturbation, including residue mutation, interchainresidue exchange, alanine scanning and chemical modification [2, 3], which isguided by structural information obtained for the protein or protein-peptidecomplex system. The key limitation of this strategy is the negligence ofpotential secondary structure changes upon sequence perturbation and howthe resulting changes may shift the protein-peptide binding structures. High-Throughput-Screening (HTS) has also been widely applied in peptide design.HTS is a brute force method to identify bioactive peptides by rapidly conduct-ing thousands to millions of biochemical, genetic, or pharmacological assays.However, HTS demands highly specialized instrumentation, development andadoption of appropriate bioassays, and high quality peptide libraries. Addi-tionally, actives discovered in HTS are often serendipitous [4]. To complementHTS, virtual screening, such as peptide docking, has been heavily used for pep-tide inhibitor design [5]. Peptide docking is usually composed of a samplingtechnique to explore peptide conformations and a scoring function to evalu-ate the strength of peptide binding for all sampled peptide binding poses. Thetwo-step protocol takes at least minutes to evaluate one protein-peptide com-plex [6], which limits its ability in sequence exploration and thus its use in denovo peptide design.

Springer Nature 2021 LATEX templateArticle Title3In recent years, machine learning-based molecular design has witnessedrapid development. A very important area in deep generative models is theefficient representation of molecules. Prior to the deep learning era, fingerprintdescriptors such as Morgan fingerprints [7] for small molecules and atom-pairfingerprints for proteins [8] were prevalent. However, these representations thatencode the chemical and structural information of individual molecules are nottask specific [9]. Deep learning models have been developed to address thislimitation by learning a unified representation in a large dataset and then finetuning this representation for a specific task. During the past few years, theresearch on molecular representation has been shifted to string transforma-tion, most of which has borrowed the idea from natural language processing.There are two major types of model framework. The first is the recurrent neu-ral network, such as Long-Short Term Memory (LSTM) [10], Gated RecurrentUnits [11] and Recurrent Attention [12]. These models have been utilized topredict molecular properties, such as solubility, toxicity [13–15]. The othertype is transformer [16] based models, which incorporate a multi-head atten-tion mechanism to process sequential data more efficiently. The transformerbased architectures such as ProteinBert [17] and ProteinTrans [18] have beenfrequently used in protein representation, and have shown great success inmultiple protein downstream classification or regression tasks (e.g. secondarystructure classification, fluorescence prediction). Recently, the graph represen-tation of molecules has gained great attention due to the graph’s ability toinclude more detailed structural and spatial information [19]. Graph neuralnetwork has been applied to process molecular graphs and perform propertyprediction [20–22].

Although the learned molecular representation has been exploited in manyproperty prediction models, few models are about molecular generation. In2017, Bjerrum used the RNN network to generate chemically valid molecules[23]. Since then, several studies have been published on the generation ofmolecules with optimized general properties such as logP, TPSA and QED [24–26], while work on de novo protein or peptide design has been scarce, and mostof it has focused on the design of a single protein with certain desired prop-erties, such as length, stability and isoelectric point [27, 28]. Design of smallmolecule or peptide therapeutics targeting a specific protein has been even lessexplored. In 2020, Das et al. proposed a method for antimicrobial drug designby using rejection sampling to search for appropriate molecular SMILES in alatent space [29]. In 2022, Castro et al. proposed a gradient based latent spacesearch method to design protein sequences against the ranibizumab antibody[30]. The two papers are most relevant to our work, albeit the first work isnot on peptide design and the second limits its application to a special caseprotein that has an existing dataset of 60,000 samples. The main challengein protein-specific peptide design is the lack of accurate binding affinity datacoupled with the immense peptide sequence space. To our knowledge, deeplearning models for protein specific peptide generation don’t yet exist.

In this paper, we propose a novel protein specific peptide generation scheme(Figure 1), called Latent Space Approximate Trajectory Collector (LSATC).We implement a GRU based Wasserstein auto encoder (WAE) [31] to obtaina 1D continuous latent space representation of peptides, as shown in Figure1(a). To circumvent the data scarcity problem, we design a feedback loopusing CMA-ES [32] to optimize the generator in a peptide latent space andthen collect samples on the generator’s trajectory. The process is illustrated inFigure 1(c). A fast feedback evaluator is crucial to the CMA-ES optimization.Since most structure-based binding evaluation (e.g., computational docking)would be computationally too costly, we train an efficient surrogate model toboost the evaluation time from minutes to microseconds (Figure 1(b)). Forthe surrogate model training, the binding energies and the hydrophobicitiesof a reasonably small number of randomly sampled peptides are evaluatedusing Pyrosetta and Biopython, respectively. Finally, we sample peptides onthe explored trajectory whose associated losses are lower than a predefinedthreshold value. We note that the increase in speed offered by the surrogatemodel enables us to fully explore the peptide encoder space for peptide designand optimization. Additionally, the incorporation of the biophysics-based sur-rogate model into our deep learning scheme also improves the interpretabilityof our LSATC model and facilitates the final peptide candidate ranking andselection.

We test our LSATC model in a multi-objective peptide extension task,which aims to improve the binding of a base peptide“YPEDILDKHLQRVIL”with beta-catenin by extending its N-terminus, while maintaining or loweringthe hydrophobicity of the extended peptides to minimize non-specific binding.For simplicity, we only consider peptide extensions of 5 amino acids, whichlimits the search space to 3.2 million peptides. However, it is worth noting thatLSATC is not limited to the generation of fixed length peptides. The entire pep-tide extension process, including the dataset generation and model training,takes two days to finish on a local 16-core CPU computer. Our LSATC modelis proven to be much more efficient in generating desired peptides than randomgeneration and several other commonly used generative models. The experi-mental results show that our generated peptides are both more potent and lesshydrophobic than the base peptide, with the highest binding improvement by3 fold.

Our contributions are:

•We build a de novo peptide generator that can efficiently generate morepotent peptide sequences targeting a specific protein when no or very littlebinding data exist.

•We design a series of samplers on an optimization trajectory in a latentspace, which not only solves the data scarcity problem in target specificpeptide design, but also improves the interpretability of the model.

•Our proposed design pipeline is highly automated and experimental testready. We combine a physics-based scoring scheme with machine learn-ing models to systematically prioritize a subset of promising peptides forexperimental testing from a pool of LSATC generated peptides.

1. P100 Nvida GPU

2. 16-cores CPU

a. Data preparation:

- labeled dataset:

1. cd 'data_generation/binding_energy_test'

2. python get_gt_dataset.py

3. put the generated data into 'LSATC/data/dataset' folder

- unlabeled dataset:

1. cd 'LSATC/data'

2. python generate_unlabeled_data.py

3. put the generated data into 'LSATC/data/dataset' folder

b. WAE model for sequence reconstruction

1. cd 'LSATC/VAE'

2. python train_vae.py

c. surrogate property prediction model

1. cd 'LSATC/prop_prediction'

2. python train_model.py

d. peptide sequence generation

- latent space search

1. cd 'LSATC/seq_sampling'

2. python CMA_ES_prop.py

- trajecotry collector

1. python trajectory_collector.py

e. other methods

- CWAE

1. cd 'LSATC/comparing_CVAE'

2. python create_dataset.py to create categorical label

3. put the categorical label dataset into 'LSATC/data/dataset' folder

4. python train_CVAE.py

- GMM

1. cd 'LSTAC/comparing_VAE_GMM'

2. python train_model.py

f. results evaluations

1. cd 'LSATC/seq_sampling'

2. you should be able to find all the analysis figures by running figure_2.py, figure_3.py, figure_4.py and support_information.py

2 days

1. WAE is properly trained so that the reconstruction loss and the miss match is close to 0

2. Property prediction network is properly trained so that on the test dataset the hydrophobicty prediction is almost perfect and the mean prediction of the binding enery is around 0.8.

3. CMA-ES based trajectory collector can sample peptides whose mean performance (hydrophobicty rank + binding energy rank) is much lower than that of the labelled dataset and the IQR is much narrower than that of the labelled dataset.

4. You shall be able to run the "results evaluations" in Procedure to get all the computational evaluation results similar to those presented in the paper.

5. We anticipate that if you perform the Gibbs Cluster on the sampled peptides from LSATC for the experiments, you will get much lower binding evaluation than that of your base peptide.

[1] Lau, J.L., Dunn, M.K.: Therapeutic peptides: Historical perspectives,current development trends, and future directions. Bioorganic and Medic-inal Chemistry26(10), 2700–2707 (2018). https://doi.org/10.1016/j.bmc.2017.06.052. Peptide Therapeutics

[2] Xiao, X., Wang, Y., Seroski, D.T., Wong, K.M., Liu, R., Paravastu, A.K.,Hudalla, G.A., Hall, C.K.: De novo design of peptides that coassembleintoβsheet-based nanofibrils. Sci Adv7(36), 7668 (2021)

[3] Eustache, S., Leprince, J., Tuff ́ery, P.: Progress with peptide scanning tostudy structure-activity relationships: the implications for drug discovery.Expert Opin Drug Discov11(8), 771–784 (2016)

[4] Lyu, W., Deng, Z., Sunkara, L.T., Becker, S., Robinson, K., Matts, R.,Zhang, G.: High throughput screening for natural host defense peptide-inducing compounds as novel alternatives to antibiotics. Frontiers inCellular and Infection Microbiology8(2018). https://doi.org/10.3389/fcimb.2018.00191

[5] Valiente, P.A., Wen, H., Nim, S., Lee, J., Kim, H.J., Kim, J., Perez-Riba,A., Paudel, Y.P., Hwang, I., Kim, K.-D., Kim, S., Kim, P.M.: Computa-tional design of potent D-Peptide inhibitors of SARS-CoV-2. J Med Chem64(20), 14955–14967 (2021)

[6] Raveh, B., London, N., Zimmerman, L., Schueler-Furman, O.: RosettaFlexPepDock ab-initio: simultaneous folding, docking and refinement ofpeptides onto their receptors. PLoS One6(4), 18934 (2011)

[7] Morgan, H.L.: The generation of a unique machine description for chemi-cal structures-a technique developed at chemical abstracts service. Journalof Chemical Documentation5(2), 107–113 (1965). https://doi.org/10.1021/c160017a018

[8] Carhart, R.E., Smith, D.H., Venkataraghavan, R.: Atom pairs as molec-ular features in structure-activity studies: definition and applications.Journal of Chemical Information and Computer Sciences25(2), 64–73(1985). https://doi.org/10.1021/ci00046a002

[9] Capecchi, A., Probst, D., Reymond, J.-L.: One molecular fingerprintto rule them all: drugs, biomolecules, and the metabolome. Jour-nal of Cheminformatics12(1), 43 (2020). https://doi.org/10.1186/s13321-020-00445-4

[10] Hochreiter, S., Schmidhuber, J.: Long short-term memory9(8) (1997).https://doi.org/10.1162/neco.1997.9.8.1735

[11] Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation ofgated recurrent neural networks on sequence modeling. In: NIPS 2014Workshop on Deep Learning, December 2014 (2014)

[12] Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K.: Recurrent models ofvisual attention. NIPS’14. MIT Press, Cambridge, MA, USA (2014)

[13] Feinberg, E.N., Sur, D., Wu, Z., Husic, B.E., Mai, H., Li, Y., Sun, S.,Yang, J., Ramsundar, B., Pande, V.S.: Potentialnet for molecular propertyprediction. ACS Central Science4(11), 1520–1530 (2018). https://doi.org/10.1021/acscentsci.8b00507

[14] Li, C., Feng, J., Liu, S., Yao, J.: A novel molecular representation learn-ing for molecular property prediction with a multiple SMILES-Basedaugmentation. Comput Intell Neurosci2022, 8464452 (2022)

[15] Paul, A., Jha, D., Al-Bahrani, R., Liao, W.-k., Choudhary, A.N., Agrawal,A.: Chemixnet: Mixed dnn architectures for predicting chemical prop-erties using multiple molecular representations. ArXivabs/1811.08283(2018)

[16] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez,A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In:Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vish-wanathan, S., Garnett, R. (eds.) Advances in Neural InformationProcessing Systems, vol. 30. Curran Associates, Inc., ??? (2017).https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

[17] Brandes,N.,Ofer,D.,Peleg,Y.,Rappoport,N.,Linial,M.:ProteinBERT:auniversaldeep-learningmodelofpro-teinsequenceandfunction.Bioinformatics38(8),2102–2110(2022)https://academic.oup.com/bioinformatics/article-pdf/38/8/2102/45474534/btac020.pdf.https://doi.org/10.1093/bioinformatics/btac020

[18] Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones,L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., Bhowmik, D., Rost,B.: ProtTrans: Toward understanding the language of life through Self-Supervised learning. IEEE Trans Pattern Anal Mach Intell44(10), 7112–7127 (2022)

[19] Guo, Z., Nan, B., Tian, Y., Wiest, O., Zhang, C., Chawla, N.V.: Graph-based Molecular Representation Learning. arXiv (2022). https://doi.org/10.48550/ARXIV.2207.04869. https://arxiv.org/abs/2207.04869

[20] Fang, X., Liu, L., Lei, J., He, D., Zhang, S., Zhou, J., Wang, F., Wu,H., Wang, H.: Geometry-enhanced molecular representation learning forproperty prediction. Nature Machine Intelligence4(2), 127–134 (2022).https://doi.org/10.1038/s42256-021-00438-4

[21] Wang, Y., Wang, J., Cao, Z., Barati Farimani, A.: Molecular con-trastive learning of representations via graph neural networks. NatureMachine Intelligence4(3), 279–287 (2022). https://doi.org/10.1038/s42256-022-00447-x

[22] Yang, K., Swanson, K., Jin, W., Coley, C., Eiden, P., Gao, H., Guzman-Perez, A., Hopper, T., Kelley, B., Mathea, M., Palmer, A., Settels, V.,Jaakkola, T., Jensen, K., Barzilay, R.: Analyzing learned molecular rep-resentations for property prediction. Journal of Chemical Informationand Modeling59(8), 3370–3388 (2019). https://doi.org/10.1021/acs.jcim.9b00237

[23] Bjerrum, E.J., Threlfall, R.: Molecular generation with recurrent neuralnetworks (rnns). ArXivabs/1705.04612(2017)

[24] Bagal, V., Aggarwal, R., Vinod, P.K., Priyakumar, U.D.: Molgpt: Molec-ular generation using a transformer-decoder model. Journal of ChemicalInformation and Modeling62(9), 2064–2076 (2022). https://doi.org/10.1021/acs.jcim.1c00600

[25] Popova, M., Isayev, O., Tropsha, A.: Deep reinforcement learn-ing for de novo drug design. Science Advances4(7), 7885(2018)https://www.science.org/doi/pdf/10.1126/sciadv.aap7885.https://doi.org/10.1126/sciadv.aap7885

[26] Wang, F., Feng, X., Guo, X., Xu, L., Xie, L., Chang, S.: Improving denovo molecule generation by embedding lstm and attention mechanism incyclegan. Frontiers in Genetics12(2021). https://doi.org/10.3389/fgene.2021.709500

[27] Wu, Z., Johnston, K.E., Arnold, F.H., Yang, K.K.: Protein sequencedesign with deep generative models. Current Opinion in Chemical Biology65, 18–27 (2021). https://doi.org/10.1016/j.cbpa.2021.04.004. Mechanis-tic Biology * Machine Learning in Chemical Biology

[28] Anand, N., Eguchi, R., Mathews, I.I., Perez, C.P., Derry, A., Altman,R.B., Huang, P.-S.: Protein sequence design with a learned poten-tial. Nature Communications13(1), 746 (2022). https://doi.org/10.1038/s41467-022-28313-9

[29] Das, P., Sercu, T., Wadhawan, K., Padhi, I., Gehrmann, S., Cipcigan,F., Chenthamarakshan, V., Strobelt, H., dos Santos, C., Chen, P.-Y.,Yang, Y.Y., Tan, J.P.K., Hedrick, J., Crain, J., Mojsilovic, A.: Acceleratedantimicrobial discovery via deep generative models and molecular dynam-ics simulations. Nature Biomedical Engineering5(6), 613–623 (2021).https://doi.org/10.1038/s41551-021-00689-x

[30] Castro, E., Godavarthi, A., Rubinfien, J., Givechian, K., Bhaskar, D.,Krishnaswamy, S.: Transformer-based protein generation with regularizedlatent space optimization. Nature Machine Intelligence (2022). https://doi.org/10.1038/s42256-022-00532-1

[31] Tolstikhin, I., Bousquet, O., Gelly, S., Sch ̈olkopf, B.: Wasserstein Auto-Encoders. In: 6th International Conference on Learning Representations(ICLR) (2018).https://openreview.net/forum?id=HkL7n1-0b

[32] Igel, C., Hansen, N., Roth, S.: Covariance matrix adaptation for multi-objective optimization. Evolutionary Computation15(1), 1–28 (2007).https://doi.org/10.1162/evco.2007.15.1.1

[33] Kyte, J., Doolittle, R.F.: A simple method for displaying the hydropathiccharacter of a protein. J Mol Biol157(1), 105–132 (1982)

[34] Chaudhury, S., Berrondo, M., Weitzner, B.D., Muthu, P., Bergman, H.,Gray, J.J.: Benchmarking and analysis of protein docking performance inrosetta v3.2. PLoS One6(8), 22477 (2011)

[35] Andreatta, M., Alvarez, B., Nielsen, M.: Gibbscluster: unsupervised clus-tering and alignment of peptide sequences. Nucleic acids research45(W1),458–463 (2017)

[36] Thomsen, M., Nielsen, M.: Seq2logo: A method for construction and visu-alization of amino acid binding motifs and sequence profiles includingsequence weighting, pseudo counts and two-sided representation of aminoacid enrichment and depletion. Nucleic acids research40, 281–7 (2012).https://doi.org/10.1093/nar/gks469

[37] Seo, S., Choi, J., Park, S., Ahn, J.: Binding affinity prediction for protein–ligand complex using deep attention mechanism based on intermolecularinteractions. BMC Bioinformatics22(1), 542 (2021). https://doi.org/10.1186/s12859-021-04466-0

[38] Wang, K., Zhou, R., Li, Y., Li, M.: DeepDTAF: a deep learn-ing method to predict protein–ligand binding affinity. Briefingsin Bioinformatics22(5) (2021) https://academic.oup.com/bib/article-pdf/22/5/bbab072/40261745/bbab072.pdf. https://doi.org/10.1093/bib/bbab072. bbab072

[39] Rezaei, M.A., Li, Y., Wu, D., Li, X., Li, C.: Deep learning in drug design:Protein-Ligand binding affinity prediction. IEEE/ACM Trans ComputBiol Bioinform19(1), 407–417 (2022)

[40] Webb, B., Sali, A.: Comparative protein structure modeling using mod-eller. Current protocols in bioinformatics54(1), 5–6 (2016)

[41] Wang, E., Sun, H., Wang, J., Wang, Z., Liu, H., Zhang, J.Z., Hou, T.:End-point binding free energy calculation with mm/pbsa and mm/gbsa:strategies and applications in drug design. Chemical reviews119(16),9478–9508 (2019)

[42] Barlow, K.A., O Conchuir, S., Thompson, S., Suresh, P., Lucas, J.E.,Heinonen, M., Kortemme, T.: Flex ddg: Rosetta ensemble-based estima-tion of changes in protein–protein binding affinity upon mutation. TheJournal of Physical Chemistry B122(21), 5389–5399 (2018)[43] Chaudhury, S., Lyskov, S., Gray, J.J.: Pyrosetta: a script-based inter-face for implementing molecular modeling algorithms using rosetta.Bioinformatics26(5), 689–691 (2010)[44] Hansen, N.: The CMA evolution strategy: A tutorial. CoRRabs/1604.00772(2016) 1604.00772[45] Auger, A., Hansen, N.: A restart cma evolution strategy with increasingpopulation size. In: 2005 IEEE Congress on Evolutionary Computa-tion, vol. 2, pp. 1769–17762 (2005). https://doi.org/10.1109/CEC.2005.1554902[46] Hansen, N., Ostermeier, A.: Completely derandomized self-adaptationin evolution strategies. Evolutionary Computation9(2), 159–195 (2001).https://doi.org/10.1162/106365601750190398[47] Andreatta, M., Lund, O., Nielsen, M.: Simultaneous alignment and clus-tering of peptide data using a Gibbs sampling approach. Bioinformatics29(1), 8–14 (2012) https://academic.oup.com/bioinformatics/article-pdf/29/1/8/17100983/bts621.pdf.

Supportinformation.docx

Download PDF

Version 1

posted

You are reading this latest protocol version

Target specific peptide design using latent space approximate trajectory collector

Status:

Version 1

Abstract

Figures

Introduction

Equipment

Procedure

Time Taken

Anticipated Results

References

Supplementary Files

Status:

Version 1

Privacy Policy

Terms of Service

Target specific peptide design using latent space approximate trajectory collector

Status:

Version 1

Abstract

Figures

Introduction

Equipment

Procedure

Time Taken

Anticipated Results

References

Supplementary Files

Status:

Version 1

Privacy Policy

Terms of Service

Manage Cookie Preferences