Design of Peptide-Guided Protein Editors via Generative Language Models

Introduction:: Curing disease is one of the greatest challenges for the future of human health, and targeted therapeutics have served as potent solutions to this problem. Small molecule inhibitors, specifically, have found significant success in the clinic, but are still limited in their therapeutic potential. Most notably, they must bind active sites, which either are not present or accessible on classically “undruggable” target proteins. Protein-based binders, such as monoclonal antibodies, are highly selective, but necessitate extensive, laborious, and cost-ineffective development cycles, hampering their scalability to different diseases. More recent computational protein design tools rely heavily on accurate structural predictions of the target protein, thus excluding disordered or unstable proteins. For this abstract, I will describe our efforts to design short binder sequences directly from the target protein sequence alone, without the requirement of a solved or predicted three-dimensional structure. Specifically, we use algorithms borrowed from natural language processing (NLP) to learn expressive, high-dimensional representations of our target protein sequences, and leverage these embeddings to generate high-affinity, selective, and unique short peptide binders to them. We then test these peptides in the wet lab, by fusing them to enzyme domains that install specific post-translational modifications, thus enabling direct, user-defined modulation of any target protein.

Materials and Methods:: Dataset Derivation

The PDB-derived dataset for our models was generated by mining the RCSB PDB for verified, high-resolution PPI structures. Every interaction of every assembly of every co-crystal in the PDB was retrieved, and then the interactions were filtered for uniqueness (a unique interaction was one with a unique pair of partners, or with significantly different ( >100 Å2) buried surface area for the same pair of partners). Filtration yielded 420,000 PPIs. Next, all interaction structures with amino acid sequence length greater than 50 and less than 1023 (for computational training speed) were processed with Rosetta PeptiDerive. extracting a list of derived peptides and their associated Rosetta energy scores (REUs), with lower scores indicating higher predicted stability.

Model Training

All models were trained in PyTorch via the HuggingFace Transformers library. Hyperparameter tuning was conducted in Optuna and different loss functions were explored, including binary cross entropy loss for SaLT&PepPr and PepGPT and a contrastive loss, based on cosine similarity, for PepPrCLIP. Models were trained until validation loss began to increase.

Experimental Testing

For screening peptides, Reporter/Target-sfGFP and Peptide-Linker-CHIP E3 ligase plasmids were cloned and transfected into HEK293T or DLD1 cells as triplicates with Lipofectamine 3000 in Opti-MEM. After 3 days post transfection, cells were harvested and analyzed on an Attune NxT Flow Cytometer for GFP fluorescence. 10,000 events were gated for data analysis. Cells expressing sfGFP were analyzed as compared to a negative control, and normalized cell fluorescence was calculated to a non-targeting control using the FlowJo software.



Results, Conclusions, and Discussions:: First, to generate peptides with predicted affinity to a target protein, we trained a SaLT&PepPr model to predict interaction sites on a given interacting partner sequence for a target protein. We based our model architecture on the ESM-2 language model, from Meta AI, which enables expressive featurization of protein sequences. We fine-tuned the final layers of ESM-2 together with a classification head, where each sequence is passed to the model with a per amino acid binary class as the target (Figure 1A). Despite not using structure as input, our model achieves superior performance to structure-based methods, suggesting strong generalization to non-homologous proteins from different families. Most importantly, when we fused model-derived peptides to E3 ubiquitin ligase domains, we demonstrate robust intracellular target degradation of undruggable, disease-causing proteins, as compared to peptides derived from AlphaFold-based methods (Figure 1B).


Next, we aimed to create a discriminator model that can capture peptide-protein selectivity, to prioritize peptide candidates that exhibit high on-target but low off-target binding. For this, we trained PepPrCLIP, which learns a latent space where correct peptide-target pairs are close together while non-interacting pairs are far apart. Leveraging ESM-3 embeddings, our final models demonstrate accurate ranking of known targeting peptides for a given target peptide candidate. Next, we search the ESM-2 latent space via a Gaussian diffusion sampler to identify novel, valid peptide sequences that can be used as input to our target-aware discriminator, providing an end-to-end generator-to-discriminator pipeline for peptide design (Figure 2A). Our laboratory results with E3 ubiquitin ligase fusions demonstrate specific and robust peptide binding and subsequent degradation.

Finally, as both SaLT&PepPr and PepPrCLIP rely on either the existence of interacting partner sequences and pre-generation of peptide sequences, we sought to train a purely de novo generation pipeline, conditioned on target sequence alone. To do this, we trained an instance of the GPT-2 decoder with peptide-protein interaction data via a fine-tuning module (Figure 2B). Our perplexities of validation peptide sequences reached a minimum of 13.2 indicating high confidence peptide generation. PepGPT-generated peptides further demonstrated effective degradation and functional phenotypic effects in cancer cell models (Figure 2C).


Acknowledgements (Optional): :

References (Optional): :