Abstract
Short-text clustering (STC) remains a critical challenge in natural language processing because of the inherent sparsity, noise, and semantic ambiguity in brief textual data. Traditional methods such as TF-IDF and static embeddings are hindered by polysemy or noise challenges. This study introduces Semantic-Aware Iterative Clustering with Adaptive Refinement (SICAR), a novel framework that redefines clustering. Four key innovations are applied: (1) contextual bidirectional encoder representations from transformer (BERT) embeddings to disambiguate polysemous terms, (2) silhouette-driven dynamic retention rates for adaptive cluster balancing, (3) density-aware outlier removal via the isolation forest algorithm to mitigate noise, and (4) iterative label refinement via a fine-tuned BERT classifier. Experiments on benchmark datasets (TMNews, SearchSnippets, Twitter, SearchSnippet-test and BioMedical) demonstrate the superiority of SICAR over other frameworks. In particular, SICAR outperforms state-of-the-art baselines by 12% in terms of clustering accuracy (CA) and 15% in terms of normalized mutual information. The robustness of SICAR is also exceptional in noisy and sparse environments. SICAR can resolve contextual ambiguities (e.g., distinguishing “Java” as programming vs. travel) and filter 85% of slang or typographical errors in social media data. These features enable precise clustering for real-world applications, such as trend detection, query disambiguation, and content moderation. Ultimately, the gap between theoretical robustness and practical scalability can be bridged in short-text analysis.