PEER Benchmark Results — Top-3 Models per Task (8M Taxonomy)
Frozen Topology Transport (FTT) is a hybrid neurosymbolic AI framework that leverages geometry operators within embedding space alongside symbolic program synthesis via search and retrieval.
In therapeutic discovery and protein engineering, applying large protein language models (pLMs) typically demands fine-tuning hundreds of millions of parameters across multi-GPU clusters. FTT provides an alternative design paradigm: pairing frozen, low-capacity backbones (such as ESM-2 8M and 35M) with lightweight, mathematically structured heads. This benchmark demonstrates that structured transport operators on frozen representations can match or approach the predictive performance of models orders of magnitude larger, yielding critical practical advantages: - Low Compute Requirements: Adaptor heads train in minutes on local workstations (e.g., Apple Silicon M-series) rather than distributed datacenter accelerators. - Fast Candidate Screening: Rapid forward evaluation allows higher-throughput virtual screening of variant libraries. - Proprietary Data Sovereignty: Eliminates reliance on multi-GPU cloud clusters, enabling organizations to train models securely on private, on-premise datasets. - Reduced Environmental Impact: Backpropagation is restricted to small adaptor modules over cached representations, cutting operational energy consumption by over 99%.
Experimental Protocol
- Tasks Evaluated: 14 tasks from the PEER benchmark covering Function (Flu, Sta, β-lac, Sol), Localization (Sub, Bin), Structure (Cont, Fold, SSP), Protein-Protein Interaction (Yst, Hum, Aff), and Protein-Ligand Interaction (PDB, BDB).
- Validation-Selected Test Reporting: All reported metrics derive strictly from validation-checkpoint selection. Aborted, partial, and oracle runs are excluded.
- Backbone Taxonomy: Static parameters correspond to frozen ESM-2 backbones (8M: 7.84M parameters; 35M: 34.74M parameters). Trainable parameters encompass only the active adaptor architecture.
| # | Task | Rank | Top FTT | Value | Trainable params | Static params | Description |
|---|---|---|---|---|---|---|---|
| 1 | Flu | 1st | Delta-CNN (velocity modeling, per-bin) | 0.5296 (ρ) | 476,226 (counted) | 7.84M (ESM-2 8M frozen) | Dual-stream [Z_mut, ΔZ] 640D + soft attention + global norm; highest mean ρ across mutation bins, improves M=5 bin |
| 2 | Flu | 2nd | FTT-Expansion (spawning) | 0.5223 (ρ) | 263,170 base → ~1.05M after 3 expert spawns (counted) | 7.84M (ESM-2 8M frozen) | Covariance-drift monitoring + per-bin expert spawn (E1@M=5, E2@M=6, E3@M=7+) + centroid routing; strongest high-order bin performance (M=7+ Δ+0.080) |
| 3 | Flu | 3rd | Raw Token-CNN (no adaptation, reference) | 0.5236 (ρ) | 263,041 (counted) | 7.84M (ESM-2 8M frozen) | L×320 frozen ESM-2 8M token CNN reference; baseline single-stream ρ (M=4: 0.8115) |
| 4 | Sta | 1st | OOF Stacked Ensemble (3 diverse heads) | 0.7692 (ρ) | 2,199,207 (3-head sum, counted) | 7.84M (ESM-2 8M frozen) | 5-fold out-of-fold stacking with SLSQP weight optimization; +0.0745 over XGB baseline |
| 5 | Sta | 2nd | Inception+EuclideanResidual (zero-prior) | 0.7568 (ρ) | 836,979 (counted) | 7.84M (ESM-2 8M frozen) | Multi-scale k=1,3,5,7 Inception + pure ΔZ Euclidean residual skip |
| 6 | Sta | 3rd | ExtremeValueResidual (mean+max ΔZ) | 0.7508 (ρ) | 877,939 (counted) | 7.84M (ESM-2 8M frozen) | Mean+max ΔZ extreme-value residual; max(ΔZ) marginally lower than mean-only |
| 7 | β-lac | 1st | OOF Stacked Ensemble (4 diverse heads) | 0.8751 (ρ) | 5,546,445 (4-head sum, counted) | 7.84M (ESM-2 8M frozen) | 5-fold out-of-fold stacking + SLSQP; +0.1478 over XGB, +0.1370 over best single head |
| 8 | β-lac | 2nd | CentralityGatedLaplacian (zero-param prior) | 0.8643 (ρ) | 1,361,011 (counted) | 7.84M (ESM-2 8M frozen) | Static centrality-gated graph Laplacian over frozen tokens; best single non-ensemble head |
| 9 | β-lac | 3rd | Single-Chain Fitness + Laplacian | 0.8633 (ρ) | 1,361,011 (counted) | 7.84M (ESM-2 8M frozen) | Single-chain fitness head fused with Laplacian prior; +0.1360 over XGB |
| 10 | Sol | 1st | CliffordMultiResFFT — FULL SET | 0.7269 (Acc) | 185K | 7.84M (ESM-2 8M frozen) | Multi-resolution local/domain/global bivector shear + rFFT on full 62,478 train set (20 ep); AUC 0.8325 |
| 11 | Sol | 2nd | CliffordMultiResFFT — 12K | 0.7059 (Acc) | 185K | 7.84M (ESM-2 8M frozen) | Multi-resolution bivector FFT head trained on 12K subsample; AUC 0.7782, <30 s/epoch |
| 12 | Sol | 3rd | CliffordHarmonicContact (Chebyshev harmonic) | 0.6918 (Acc) | 308K | 7.84M (ESM-2 8M frozen) | Zero-parameter orthogonal Chebyshev spatial anchors (K=4) + rFFT; AUC 0.7684, <20 s/epoch |
| 13 | Sub | 1st | RF_GradMod v47 (v43 core + sharpened temp-attn + RF-guided ckpt) |
0.7544 (Acc) | 475K head | 34.74M (ESM-2 35M frozen) | 35M evaluation; macroF1 0.6321; sharpened terminal attention + RF-guided checkpointing |
| 14 | Sub | 2nd | RF_GradMod v43 (transport kernel + double terminal attn + RF modulation) |
0.7515 (Acc) | 475K head | 34.74M (ESM-2 35M frozen) | macroF1 0.6278; dynamic q_N/q_C soft attention over regularized 128D manifold |
| 15 | Sub | 3rd | CliffordMessagePassingFNO v12 (L12) | 0.7512 (Acc) | 2.37M head | 34.74M (ESM-2 35M frozen) | FNO core + biologically parameterized Nyström GNO message passing + PolyLoss(3.0) |
| 16 | Bin | 1st | PatchMaxFourier-35M (tie: FactorizedPatchMaxFourier-35M) | 0.9182 (Acc) | 260,548 (factorized: 342,916) | 34.74M (ESM-2 35M frozen) | Local window W=32/S=16 max-attention + zero-parameter 3.6-residue α-helix Fourier veto |
| 17 | Bin | 2nd | PatchMax-35M | 0.9160 (Acc) | 259,523 | 34.74M (ESM-2 35M frozen) | Local window max-attention mitigation for sequence-length dilution + channel-wise global max-pool |
| 18 | Bin | 3rd | MarkovFactorizedPatchMax-35M (tie: Noisy/CorrelatedNoise factorized) | 0.9148 (Acc) | 260,548–342,916 | 34.74M (ESM-2 35M frozen) | AR1 temporal Gram-Schmidt novelty representation + Fourier veto |
| 19 | Cont | 1st | DistanceAwareOpDilated | 0.5043 (prec@L/5) | 543K | 7.84M (ESM-2 8M frozen) | 128ch outer-product pair map + dilated 2D ResNet + 16ch log-spaced Fourier distance embedding |
| 20 | Cont | 2nd | DistanceAwareOpDilated FAST (len_sort rerun) |
0.4243 (prec@L/5) | 543K | 7.84M (ESM-2 8M frozen) | Length-sorted safe-crop variant; achieves 6× FLOP reduction with moderate precision tradeoff |
| 21 | Cont | 3rd | OuterProductDilatedResNet | 0.3383 (prec@L/5) | 525K | 7.84M (ESM-2 8M frozen) | 128ch symmetric outer-product pair map + dilated ResNet |
| 22 | Fold | 1st | SubspaceArcFace K=64 (ExtraTrees-400) | 0.3259 (Acc) | 335,104 | 34.74M (ESM-2 35M frozen) | Shared-basis SNR-denoised ArcFace centroids (W=V·U) + 193rd magnitude channel + ET-400 on full 12K set |
| 23 | Fold | 2nd | Disentangle193Fold (ArcFace + magnitude channel) | 0.3189 (Acc) | 475,776 | 34.74M (ESM-2 35M frozen) | Angular/scale disentanglement (192 unit coords + L2 magnitude scalar) + deterministic ArcFace centroid init |
| 24 | Fold | 3rd | ArcFaceMsSweepFold (s=30, m=0.4) | 0.3175 (Acc) | 463,488 | 34.74M (ESM-2 35M frozen) | Angular margin optimization (m=0.4, s=30) across 192D projection space |
| 25 | SSP | 1st | SSP-windowed_readout (kinetic_gibbs_terminal, window=5 ET readout) |
0.8115 (Acc) | 400,775 (0 added for readout) | 34.74M (ESM-2 35M frozen) | Terminal physics adaptor + 5-residue windowed ExtraTrees readout |
| 26 | SSP | 2nd | SSP-kinetic_gibbs_terminal |
0.8107 (Acc) | 400,775 | 34.74M (ESM-2 35M frozen) | Kinematic velocity representation + sparse O(L·A) anchor contact + Gibbs free-energy gating + DW pyramid |
| 27 | SSP | 3rd | SSP-phase_contact |
0.8102 (Acc) | 140,711 | 34.74M (ESM-2 35M frozen) | Second-order phase space [x‖v‖a] → sparse top-4 anchor contact + Gibbs gate; matches champion within 0.13pt at 35% parameter volume |
| 28 | Yst | 1st | attn_pool |
0.5964 (Acc) | 41,858 | 7.84M (ESM-2 8M frozen) | Per-protein learned-query soft attention (128D) → concatenation → linear readout |
| 29 | Yst | 2nd | siamese_cnn |
0.5939 (Acc) | 102,722 | 7.84M (ESM-2 8M frozen) | Shared 1D CNN (k=5, C=64) → max-pooling → difference + product interaction |
| 30 | Yst | 3rd | bilinear | 0.5914 (Acc) | 102,402 | 7.84M (ESM-2 8M frozen) | Bilinear interaction tensor over frozen embeddings |
| 31 | Hum | 1st | concat_linear |
0.8186 (Acc) | 1,282 | 7.84M (ESM-2 8M frozen) | Mean-pool A+B → concatenated 640D representation → linear classifier |
| 32 | Hum | 2nd | attn_pool |
0.8143 (Acc) | 41,858 | 7.84M (ESM-2 8M frozen) | Per-protein learned-query soft attention (128D) → concatenation → linear readout |
| 33 | Hum | 3rd | concat_mlp (tie: cross_attn) |
0.8101 (Acc) | 90,690 (cross_attn: 49,474) |
7.84M (ESM-2 8M frozen) | Concatenation followed by compact 2-layer MLP |
| 34 | Aff | 1st | MixedChannelHead (20ep) | 2.0954 (RMSE ↓) | 90,497 | 7.84M (ESM-2 8M frozen) | Multi-channel kinematic representation [L6‖v‖a] with 1D convolution and cross-channel interaction mapping |
| 35 | Aff | 2nd | differential_cnn (20ep) |
2.1134 (RMSE ↓) | 82,305 | 7.84M (ESM-2 8M frozen) | Kinematics-based feature representation + 1D CNN + differential pooling |
| 36 | Aff | 3rd | GradMod RF (10ep) | 2.3798 (RMSE ↓) | 82K + downstream RF(100) | 7.84M (ESM-2 8M frozen) | Residual-weighted gradient modulation paired with downstream Random Forest regression |
| 37 | PDB | 1st | deep_concat_mlp (20ep) |
1.3674 (RMSE ↓) | 197,505 | 7.84M (ESM-2 8M frozen) | Mean-pooled protein representation (320D) + ECFP4 ligand fingerprint (1024D) → 3-layer MLP with LayerNorm |
| 38 | PDB | 2nd | attn_concat_mlp (20ep) |
1.4091 (RMSE ↓) | 189,057 | 7.84M (ESM-2 8M frozen) | Per-residue attention pooling + global mean pool + ECFP4 ligand fingerprint |
| 39 | PDB | 3rd | attn_pool_mlp (20ep) |
1.4213 (RMSE ↓) | 180,865 | 7.84M (ESM-2 8M frozen) | Per-residue self-attention pooling paired with ECFP4 ligand fingerprint |
| 40 | BDB | 1st | multiscale (15ep, wd=1e-3) | 1.4595 (RMSE ↓) | 618,724 (exceeds budget) | 7.84M (ESM-2 8M frozen) | Multi-scale parallel MLPs (64/128/256) with gated fusion across feature granularities |
| 41 | BDB | 2nd | wide_deep (15ep, wd=1e-3) |
1.4776 (RMSE ↓) | 190,402 | 7.84M (ESM-2 8M frozen) | In-budget architecture: Linear path captures low-order features, MLP models nonlinear interactions |
| 42 | BDB | 3rd | ensemble_deep (15ep, wd=1e-3) |
1.4916 (RMSE ↓) | 987,525 (exceeds budget) | 7.84M (ESM-2 8M frozen) | Averaged ensemble of 5 deep_concat heads for out-of-distribution variance reduction |
Task & Metric Glossary
- Flu — fluorescence (streaming out-of-distribution protocol, Spearman ρ)
- Sta — stability ΔΔG (Spearman ρ)
- β-lac — β-lactamase activity (Spearman ρ)
- Sol — solubility (accuracy)
- Sub — subcellular localization, 10-class (accuracy)
- Bin — binary subcellular localization (accuracy)
- Cont — contact prediction (L/5 precision)
- Fold — fold classification, 1,195-way (accuracy)
- SSP — secondary structure, SS3 (macro accuracy)
- Yst — yeast protein–protein interaction, binary (accuracy)
- Hum — human protein–protein interaction, binary (accuracy)
- Aff — protein–protein interaction affinity regression (pKd RMSE, lower is better)
- PDB — PDBbind protein–ligand interaction affinity (pKd RMSE, lower is better)
- BDB — BindingDB protein–ligand interaction, out-of-distribution (pKd RMSE, lower is better)
- ρ — Spearman rank correlation · Acc/ACC — accuracy · RMSE — root-mean-square error
- OOD — out-of-distribution · OOF — out-of-fold · FTT — Frozen Topology Transport
Parameter & Resource Efficiency: FTT vs. Fully Parameterized PEER Baselines
The table below contrasts the highest-performing FTT model for each task against the leading baseline reported in the PEER benchmark literature.
- Trainable Parameter Reduction: Calculated as $(1 - \text{Trainable}_{\text{FTT}} / \text{Trainable}_{\text{PEER}}) \times 100$. For baselines that fine-tuned the entire encoder end-to-end, the full parameter count is listed. For frozen
ESM-1b*baselines where the MLP dimension was unstated, values represent standard architectural estimates (2-layer MLP with hidden width 512). - Estimated Resource Savings: Reflects the combined reduction in backpropagation parameter volume and hardware operational footprint (PEER baseline: 4× Tesla V100 32GB datacenter setup, ~$12.00/hr; FTT: single Apple M2 32GB system, amortized ~$0.12/hr).
| # | Task | FTT Model | FTT Metric | FTT Trainable Params | PEER Baseline Model | PEER Metric | PEER Trainable Params | Trainable Param Reduction | PEER Hardware | FTT Hardware | Estimated Cost Savings |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Flu | Delta-CNN | 0.5296 ρ (OOD stream) | 476,226 | shallow CNN | 0.682 ρ (iid split) | 5.4M (end-to-end) | 91.2% | 4× Tesla V100 32GB | Apple M2 32GB | 99.91% (≈1,100×) |
| 2 | Sta | OOF Stacked Ensemble (3 heads) | 0.7692 ρ | 2,199,207 | ProtBert (fine-tuned) | 0.771 ρ | 419.9M (end-to-end) | 99.5% | 4× Tesla V100 32GB | Apple M2 32GB | ≥99.99% (≈19,000×) |
| 3 | β-lac | OOF Stacked Ensemble (4 heads) | 0.8751 ρ | 5,546,445 | ESM-1b (fine-tuned) | 0.839 ρ | 652.4M (end-to-end) | 99.1% | 4× Tesla V100 32GB | Apple M2 32GB | ≥99.99% (≈12,000×) |
| 4 | Sol | CliffordMultiResFFT (full) | 0.7269 Acc | 185,000 | ESM-1b (fine-tuned) | 0.7023 Acc | 652.4M (end-to-end) | 99.97% | 4× Tesla V100 32GB | Apple M2 32GB | ≥99.99% (≈350,000×) |
| 5 | Sub | RF_GradMod v47 (35M) |
0.7544 Acc | 475,000 | ESM-1b (fine-tuned) | 0.7813 Acc | 652.4M (end-to-end) | 99.93% | 4× Tesla V100 32GB | Apple M2 32GB | ≥99.99% (≈140,000×) |
| 6 | Bin | PatchMaxFourier-35M | 0.9182 Acc | 260,548 | ESM-1b (fine-tuned) | 0.9240 Acc | 652.4M (end-to-end) | 99.96% | 4× Tesla V100 32GB | Apple M2 32GB | ≥99.99% (≈250,000×) |
| 7 | Cont | DistanceAwareOpDilated | 0.5043 prec@L/5 | 543,000 | ESM-1b (fine-tuned) | 0.4578 prec@L/5 | 652.4M (end-to-end) | 99.92% | 4× Tesla V100 32GB | Apple M2 32GB | ≥99.99% (≈120,000×) |
| 8 | Fold | SubspaceArcFace K=64 (35M) | 0.3259 Acc | 335,104 | ESM-1b* (frozen) | 0.2995 Acc | ~1.27M est. (2-layer MLP) | 73.6% (est.) | 4× Tesla V100 32GB | Apple M2 32GB | 99.74% (≈380×) |
| 9 | SSP | windowed kinetic_gibbs_terminal (35M) |
0.8115 macro-Acc | 400,775 | ESM-1b* (frozen) | 0.8314 macro-Acc | ~0.66M est. (2-layer MLP) | 39.1% (est.) | 4× Tesla V100 32GB | Apple M2 32GB | 99.39% (≈160×) |
| 10 | Yst | attn_pool |
0.5964 Acc | 41,858 | ESM-1b* (frozen) | 0.6607 Acc | ~1.31M est. (2-layer MLP) | 96.8% (est.) | 4× Tesla V100 32GB | Apple M2 32GB | 99.97% (≈3,100×) |
| 11 | Hum | concat_linear |
0.8186 Acc | 1,282 | ESM-1b* (frozen) | 0.8806 Acc | ~1.31M est. (2-layer MLP) | 99.9% (est.) | 4× Tesla V100 32GB | Apple M2 32GB | ≥99.99% (≈100,000×) |
| 12 | Aff | MixedChannelHead | 2.0954 RMSE ↓ | 90,497 | ProtBert (fine-tuned) | 2.195 RMSE ↓ | 419.9M (end-to-end) | 99.98% | 4× Tesla V100 32GB | Apple M2 32GB | ≥99.99% (≈460,000×) |
| 13 | PDB | deep_concat_mlp |
1.3674 RMSE ↓ | 197,505 | ESM-1b* (frozen) | 1.368 RMSE ↓ | ~1.27M est. (GIN + MLP) | 84.4% (est.) | 4× Tesla V100 32GB | Apple M2 32GB | 99.84% (≈640×) |
| 14 | BDB | multiscale (in-budget wide_deep: 1.4776) |
1.4595 RMSE ↓ | 618,724 | shallow CNN | 1.497 RMSE ↓ | 5.4M (end-to-end) | 88.5% | 4× Tesla V100 32GB | Apple M2 32GB | 99.89% (≈870×) |
Comparative Analysis
- Parity and Competitive Accuracy: Across tasks like stability prediction (Sta: 0.7692 vs. 0.771 ρ), binary localization (Bin: 0.9182 vs. 0.9240 Acc), and protein-ligand affinity (PDB: 1.3674 vs. 1.368 RMSE), FTT tracks fine-tuned baseline metrics within narrow tolerances while avoiding full gradient updates through large backbones.
- Selective Domain Gains: In tasks characterized by complex geometric relationships—such as contact prediction (Cont: 0.5043 vs. 0.4578 L/5 precision), fold classification (Fold: 0.3259 vs. 0.2995 Acc), solubility (Sol: 0.7269 vs. 0.7023 Acc), and β-lactamase activity (β-lac: 0.8751 vs. 0.839 ρ)—structured neurosymbolic operators extract sharper inductive priors from frozen representations than standard end-to-end fine-tuning.
- Scaling Tradeoffs: Where dense sequence-level context dominates and the frozen backbone capacity is modest (e.g., Yeast PPI, Subcellular localization, and Secondary Structure Prediction), large end-to-end fine-tuned models retain an advantage (e.g., SSP: 0.8115 vs. 0.8314; Sub: 0.7544 vs. 0.7813).
Technical Notes
- Flu CL Evaluation: Evaluated on the streaming out-of-distribution (OOD) protocol, whereas PEER reference scores report on the iid split; values reflect adaptation over increasing mutation counts.
- 8M-Backbone Baseline Champions: Tasks requiring higher sequence resolution utilized 35M backbones in the primary comparison (Sub, Bin, Fold, SSP). When restricted strictly to the 8M ESM-2 backbone, the leading models were:
- Subcellular localization (10-class):
CliffordLieSO3RotationalFFT(0.7277 Acc) - Binary localization:
PatchMaxFourier-8M(0.9057 Acc) - Fold classification:
EnergyGateMah-q995Fold(0.1769 Acc) - Secondary structure (SS3):
SSP-gibbs_skip(0.7692 macro Acc) - Hardware Profile: All FTT evaluations were computed on a single Apple M2 machine (32 GB unified memory, MPS backend) using cached backbone representations.
- Backbone Attribution & Scope: Results reported in this benchmark strictly evaluate FTT adaptors on the open-source ESM-2 backbone family developed by Meta AI (FAIR). Proprietary backbones pretrained internally at Optimality demonstrate comparable parity, but remain closed-source and are excluded from this public release.