keyword
temporal contrastive loss
Temporal contrastive loss is an objective function used in self-supervised machine learning to train neural networks on sequential data, such as time series and video, by leveraging time-based relationships. It operates by optimizing an embedding space where representations originating from the same timestamp, adjacent temporal neighborhoods, or differently augmented views of the same temporal context are pulled closer together as positive pairs, while representations from distinct timestamps or separate temporal segments are pushed farther apart as negative pairs. By enforcing consistency across local contexts while preserving discriminative differences across time, this loss enables models to capture underlying dynamics, contextual dependencies, and fine-grained temporal structures without requiring human annotations.
2 items
TS2Vec: Towards Universal Representation of Time Series
Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, Bixiong Xu
Why you should read this
Proposes a universal contrastive learning framework that uses hierarchical contrasting and contextual consistency to learn multiscale time series representations, achieving state-of-the-art results across classification, forecasting, and anomaly detection benchmarks.
This paper presents TS2Vec, a universal framework for learning representations of time series in an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature. The source code is publicly available at https://github.com/yuezhihan/ts2vec.
Added
2026-09-26
Neural Computers
Mingchen Zhuge, Changsheng Zhao, Haozhe Liu, Zijian Zhou, Shuming Liu, Wenyi Wang, Ernie Chang, Gael Le Lan, Junjie Fei, Wenxuan Zhang, Yasheng Sun, Zhipeng Cai, Zechun Liu, Yunyang Xiong, Yining Yang, Yuandong Tian, Yangyang Shi, Vikas Chandra, Jürgen Schmidhuber
Why you should read this
Proposes a revolutionary "Neural Computer" paradigm that unifies computation, memory, and I/O within a learned runtime state, offering a path beyond conventional computing architectures, agents, and world models.
We propose a new frontier: Neural Computers (NCs) -- an emerging machine form that unifies computation, memory, and I/O in a learned runtime state. Unlike conventional computers, which execute explicit programs, agents, which act over external execution environments, and world models, which learn environment dynamics, NCs aim to make the model itself the running computer. Our long-term goal is the Completely Neural Computer (CNC): the mature, general-purpose realization of this emerging machine form, with stable execution, explicit reprogramming, and durable capability reuse. As an initial step, we study whether early NC primitives can be learned solely from collected I/O traces, without instrumented program state. Concretely, we instantiate NCs as video models that roll out screen frames from instructions, pixels, and user actions (when available) in CLI and GUI settings. These implementations show that learned runtimes can acquire early interface primitives, especially I/O alignment and short-horizon control, while routine reuse, controlled updates, and symbolic stability remain open. We outline a roadmap toward CNCs around these challenges. If overcome, CNCs could establish a new computing paradigm beyond today's agents, world models, and conventional computers.
Added
2026-04-16
License
Published with permission