Developing Foundation Models for Time Series Data

In this project, we aim to pioneer foundational models specifically designed for time series data—a critical step forward in handling vast and complex temporal datasets generated across domains like healthcare, finance, environmental monitoring, and beyond. While recent advancements in foundation models have shown tremendous success in NLP and computer vision, the unique characteristics of time series data, such as temporal dependencies and lack of rich semantic make it challenging to leverage these models directly for time series tasks.

Building upon our exisiting research [1], our objective is to develop a versatile time series foundation model that can capture complex temporal patterns and adapt to various downstream tasks such as anomaly detection [2] and classification [3,4]. The ultimate goal is to create a model that, once trained on a broad and diverse set of time series data, can transfer knowledge effectively across tasks, enabling faster and more accurate insights without needing extensive retraining.

[1] Mohammadi Foumani, N., Mackellar, G., Ghane, S., Irtza, S., Nguyen, N., & Salehi, M. (2024, August). Eeg2rep: enhancing self-supervised EEG representation through informative masked inputs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 5544-5555).

[2] Darban, Z. Z., Webb, G. I., Pan, S., Aggarwal, C. C., & Salehi, M. (2025). CARLA: Self-supervised contrastive representation learning for time series anomaly detection. Pattern Recognition, 157, 110874.

[3] Foumani, N. M., Tan, C. W., Webb, G. I., & Salehi, M. (2024). Improving position encoding of transformers for multivariate time series classification. Data Mining and Knowledge Discovery, 38(1), 22-48.

[4] Foumani, N. M., Tan, C. W., Webb, G. I., Rezatofighi, H., & Salehi, M. (2024). Series2vec: similarity-based self-supervised representation learning for time series classification. Data Mining and Knowledge Discovery, 1-25.