Abstract
Abstract
Protein–protein interaction (PPI) predictors are often evaluated with random pair splits that permit proteins to recur across training and test sets, while constructed negative pairs can introduce a separate source of benchmark-specific signal. We reassessed the Guo yeast benchmark using a matched protein-novelty curve, independently generated MMseqs2 sequence-cluster-controlled R3 variants, topology-only controls, degree- and length-matched putative negatives, handcrafted and frozen protein-language-model representations, multiple model configurations, seed analyses, and connected-component-aware inference. In the matched R1→R2→R3 comparison, AAC+CTD–MLP ROC-AUC declined from 0.697 for seen–seen pairs to 0.626 for one-unseen pairs and 0.587 for both-unseen pairs. Across independently regenerated cluster-disjoint R3 variants at 50%, 40%, 30%, and 20% MMseqs2 clustering thresholds, performance showed no consistent monotonic trend. Guo original-negative endpoints had lower positive-network degree than positive endpoints (mean 3.34 versus 6.72), and a degree-only logistic model reached ROC-AUC 0.962 on the original negatives but 0.502 on degree-matched negatives. We used degree-matched negatives consistently for R3 training and testing; six sequence-based model configurations yielded ROC-AUC 0.493–0.547; none showed corrected connected-component-aware evidence of above-chance discrimination. A PIPR multispecies analysis reproduced degree-associated label separation but also exposed cross-species pairing as an additional pooled cue. These results show that protein novelty, operational sequence-cluster separation, and negative-set topology are distinct dimensions of PPI benchmark evaluation. The Guo result is benchmark-specific and should not be generalized to sequence-based PPI prediction or protein language models overall. The dataset and source code for the experiments are publicly available at
https://github.com/morufwork/guoAudit.git
.
Key Points
Random pair splits, protein-disjoint splits, sequence-cluster controls, and negative-set controls answer different evaluation questions and should be reported separately.
In Guo, AAC+CTD–MLP ROC-AUC declined from 0.697 at R1 to 0.587 at R3, while independently generated R3-50/40/30/20 variants showed no monotonic threshold trend.
Guo original negatives carried strong positive-network-degree imbalance; a degree-only logistic model achieved ROC-AUC 0.962 on N0 but 0.502 on degree-matched N2.
Under consistent R3/N2 evaluation, six sequence-model configurations had ROC-AUC 0.493–0.547, and the secondary PIPR analysis showed that pooled degree matching can introduce an additional taxonomic-pairing cue.