Protein language models (PLMs), like all machine learning models, learn biases from data. In this talk I will show that PLMs unintentionally learn a strong species bias. Specifically, PLM likelihoods of protein sequences from certain species (e.g., human, E. coli) are systematically higher, independent of the protein in question. I trace this bias' origins and demonstrate how it can be detrimental for some protein design applications, such as enhancing thermostability.

We developed Baseline Antibody Language Models (BALM) using Jaffe's dataset of 1.6 million natively paired human antibody sequences. Training with paired sequences (BALM-paired) outperformed unpaired training, demonstrating learning of cross-chain immunological features. ESM-2, a protein language model, showed similar improvements when fine-tuned with paired data. This approach addresses limitations in current antibody models, enhancing our understanding of antibody structure-function relationships. We discuss implications for antibody engineering and therapeutic development.

Align to Innovate, a non-profit research organization, is on a mission to shepherd biology into a data-first discipline through reproducible, scalable, and sharable experimentation. We run a suite of programs that work in conjunction to develop automated wet-lab experimental methods accessible to the community, collect large-scale public protein engineering datasets, and benchmark predictive and generative protein design algorithms. All our work is community-driven, collaborative, and operates under open science principals.

Several in silico metrics have been proposed as a means of assessing antibody design strategies. While these metrics have been utilized to evaluate and benchmark models, there has been little in vitro validation to determine the validity of such metrics. We showcase experiments designed to assess whether or not ranking antibodies by these metrics increases binding rates or binding affinities.

Antibodies are an important class of medicines, whose efficacy is driven by specific target binding. Given the therapeutic relevance, there have been multiple attempts to predict antibody-antigen binding affinity computationally. I will discuss our findings on how training data influences the success and selection of machine learning strategies to tackle this challenge, ranging from antigen-specific to generalizable and zero-shot affinity prediction.

The structural complex of a protein-protein interaction (PPI) can yield important mechanistic insights that support drug discovery efforts. Rigid body docking and predictive models such as AlphaFold multimer remain poor quality for difficult but clinically significant systems. Here we present AFInjection, a framework for generating and incorporating experimental data to AlphaFold to improve complex prediction. AFInjection uses affinity data from in vitro directed coevolution of a PPI, finding novel functional sequence pairs which are incorporated into AlphaFold’s features to better infer the parental complex. We demonstrate the utility of this method on antibody-antigen systems and weak PPIs with disordered regions.

Protein aggregation impacts neurodegenerative diseases and biotherapeutic manufacturing. Big data tools like AlphaFold and protein language models excel in biology but rely heavily on high-quality training datasets. This work introduces the tripartite β-lactamase assay, a novel method for generating large, high-quality datasets that link protein aggregation to cell survival, enabling deeper insights into protein behavior and aggregation-related challenges.

The majority of antibodies in the PDB targeting the same binding site as some other non-antibody proteins are mimic antibodies- they share with the other protein a motif composed of key residues at conserved geometrical positions. By investigating mimic antibodies and how they imitate the binding site of other proteins, we develop a method for identifying them in repertoire data and validate it on IL-18RA.

Classic developability assays are low-medium throughput and require complex reagent generation. The large, normalized datasets required for ML tool building require adapting or replacing these assays with ones amenable to high throughput automation. We describe our process and results for validating replacement HT developability assays and a HT developability package compatible with very early HT screening.

• Amplification strategies
• Avoiding bias
• Closed-loop experimentation
• Controls and validation
• Dealing with skewed data
• Historical data
  

• Quantity: Availability challenges, scaling laws, synthetic data
• Quality: Diversity, leakage, reproducibility, quality vs. quantity
• Collaborative data generation: Industry-academia partnerships, data sharing consortia
• Federated learning: Technical challenges, open-source foundation models
• Intellectual property: Data ownership, balancing openness with commercial interests
• Open-source data: Curation quality, integrating diverse sources with proprietary data

Scaling laws estimate over a trillion species exist, yet less than 0.00001% have been studied. Powered by a global metagenomic data supply chain, BaseFold offers improved protein structure prediction with increased accuracy, achieving up to 80% reductions in RMSD values. Leading to more reliable predictions, better docking results, and advancements in therapeutic development, all while incentivizing biodiversity protection.

Model-based prediction of biologics developability properties will increase speed to clinic. Previous pipeline program data is a valuable data source for training models but requires data curation, contextualization, annotation, and quality control for this new use. I will describe how we incorporated historical data using data quality control protocols to create reusable data products for machine learning prediction of key attributes, including hydrophobicity and polyspecificity of monoclonal antibodies.

The biophysical characterisation of biologics requires significant wet-lab resources. To enable large-scale predictions of millions of molecules, protein-language models have become an attractive proposition to accurately predict lab readouts. However, current machine-learning models are limited in accuracy largely due to lack of high-quality and high-volume training data. In this talk, we present the generation of a maximally informative dataset for the purpose of training machine-learning models for nanobody developability predictions.

T-cell engagers (TCEs) promise breakthroughs in the treatment of solid tumors, but their progression in the clinic is limited by on-target, off-tumor toxicity. In this talk, I describe how our platform integrates active learning, automation, and high-throughput functional assays to efficiently identify highly selective and potent TCEs. I highlight our utilization of the design-build-test-learn ecosystem to generate high-quality data that powers our machine learning models and therapeutic assets.