Anytime accuracy is an evaluation metric in online and continual machine learning that measures a model's predictive performance at arbitrary points throughout the learning process, rather than only after training completes or at predefined task boundaries. In streaming data scenarios where distributions shift and new classes are introduced incrementally, models are often required to handle inference requests at any moment while still actively updating their parameters. By assessing accuracy periodically or continuously during training, this metric captures how well a model balances real-time adaptation to newly observed information with the retention of previously learned knowledge, exposing transient performance drops and catastrophic forgetting that end-of-training evaluations might overlook.
