Abstract
AI agents are increasingly expected to interact with changing users, tools, and environments over extended periods of time. However, current systems remain largely static: their parameters are fixed during deployment, while new experiences, domain knowledge, corrections, and behavioral adaptations are handled primarily through external memory management and in-context learning. In this position paper, we focus on lifelong AI: AI systems that can continually learn from their long-term experience throughout an extended deployment lifetime. This position paper argues that existing in-context mechanisms are insufficient for lifelong AI, and advocate In-Parameter Learning (IPL) as a necessary complement to In-Context Learning (ICL), referring to mechanisms that allow a deployed model to continually and safely update its parameters during use. More broadly, we aim to renew the community’s attention to in-parameter learning and to recalibrate the overly optimistic expectations placed on in-context learning alone. Our central claim is that future lifelong AI systems should be built around a hybrid paradigm: in-context learning for immediate, provisional, and reversible information, and in-parameter learning for durable, cumulative, and generalizable growth.