Abstract
Joint-Embedding Predictive Architectures (JEPAs) can support model-predictive control by rolling out candidate actions in latent space, and scoring their predicted outcomes against a goal. However, such planners do not by themselves provide a natural interface for semantic preferences specified in language. We introduce \emph{Jev-JEPA}, a probabilistic control framework in which Jev, a typed probabilistic decision model, provides a zero-shot, language-conditioned semantic prior for planning with a JEPA world model. Jev assigns probabilities to high-level decision modes, which are lifted by state-conditioned trajectory proposals into a multimodal prior over continuous action sequences. This prior is combined with a JEPA predictive expert through a product-of-experts decision distribution, so language shapes \emph{where} predictive planning searches while the world model determines which candidate plans are physically plausible. The construction admits an exact KL-regularized variational characterization, an equivalent effective-energy objective, and a posterior-odds interpretation of the semantic--physical trade-off. We further introduce support smoothing and uncertainty-adaptive semantic strength, derive finite-budget sampling and combined-expert perturbation bounds, and give importance-sampling, CEM, and MPPI realizations. Jev-JEPA thereby separates goal specification, semantic preference, and predictive dynamics, allowing new language instructions to reshape decision priors without retraining or modifying the world model.