Abstract
Forecasting financial market movements requires integrating numerical time-series data with semantic information from financial narratives that shape investor behavior. While deep learning has advanced time-series prediction, existing methods often exploit trivial price autocorrelation rather than capturing genuine predictive signals from multimodal sources. We propose the Semantic-Time Fusion Architecture (STFA), which integrates structured market data with GPT-4-supervised FinBERT semantic representations of financial news through three key innovations: delay-aware temporal alignment between text publication and market reaction, semantic-guided attention mechanisms emphasizing news-relevant historical patterns, and multi-scale temporal fusion with multi-objective training to balance accuracy and interpretability. We evaluate our model on a large-scale financial news and stock price dataset using GPT-4 sentiment annotation across stratified high-liquidity stocks with strictly chronological train-validation-test splits (2013–2020 for training, 2021 for validation, 2022–2023 for testing). Sentiment annotations were generated independently for each article without using any future market information, ensuring strict temporal separation and preventing information leakage. Experiments demonstrate that STFA significantly outperforms traditional time-series models, multimodal baselines, and random walk benchmarks on return prediction, where price autocorrelation vanishes, rather than relying on price-level forecasting. Statistical validation with bootstrap confidence intervals, effect size analysis, and Bonferroni-corrected significance tests confirms robust improvements across independent runs. Ablation studies reveal that GPT-4-supervised FinBERT semantic representations provide superior signals compared with lexicon-based sentiment, with each architectural component contributing significantly to performance. This work demonstrates how temporally aligned semantic understanding enhances both predictive accuracy and model interpretability in financial forecasting.