Abstract
Drug-safety systems are built for the head of the distribution: common, anticipated adverse events that trials list in advance and frequency statistics rank first. The harms that matter most after wide use often lie in the tail—rare, unanticipated, or confined to vulnerable individuals. In 2023, clinicians saw three patients taking semaglutide with non-arteritic anterior ischemic optic neuropathy (NAION), a rare cause of permanent vision loss, within one week. We propose long-tail pharmacovigilance, a research program that makes the tail the primary target of safety evaluation, and RARE-Bench, its first benchmark, which tests whether language models can rank uncommon drug harms for clinical review. This pilot tests ranking only. Across 60 drugs, frequency baselines retrieved almost none of the low-prevalence label terms (0–0.5%); the model retrieved 3.8–4.3%. Against similar-drug transfer (2.9%), the model's edge was unresolved overall (+0.013, 95% CI −0.018 to +0.039) but differed by neighborhood (+0.034 without same-class neighbors, −0.026 with; interaction p = 0.038; exploratory). A class-only control matched the name tier (0.034 versus 0.034), so the tail advantage cannot be separated from class-level knowledge. No fabricated terms appeared among 40 audited candidates. For the case drugs, free ranking never mentioned the eye. Eye-directed questioning returned eye terms for case and negative-control drugs alike; only a reasoning model with marginal exposure named NAION itself. Tail retrieval is therefore measurable, but the pilot does not show that the method predicts new harms, benefits from paired molecular profiles, or improves trial decisions.