12-04, 18:00–18:30 (UTC), Data/ Data Science Track
Transformers are everywhere: NLP, Computer Vision, sound generation and even protein-folding. Why not in forecasting? After all, what ChatGPT does is predicting the next word. Why this architecture isn't state-of-the-art in the time series domain?
In this talk, you will understand how Amazon Chronos and Salesforece's Moirai transformer-based forecasting models work, the datasets used to train them and how to evaluate them to see if they are a good fit for your use-case.
This talk provides practical insights for Generative AI and ML practitioners interested in understanding how transformer-based architectures have been adapted to forecasting problems, and how to evaluate these models for specific use-cases.
We begin by exploring the use-cases where transformer-based forecasters excel in the time series domain. We'll introduce the concept of foundational models for forecasting (TSFM) and highlight how they differ from traditional forecasting approaches.
Next, we discuss the issue of data availability. We'll examine the current state of open datasets for training TSFMs and discuss the role of synthetic data in overcoming data scarcity challenges.
The talk then provides a brief overview of state-of-the-art TSFMs, with a particular focus on Chronos by AWS and TimesFM by Google. We analyse their architectures, methods for discretizing continuous variables, data mixing strategies, and key findings from ablation studies.
Finally, we outline future research directions for advancing TSFMs and provide guidance on conducting your own benchmarks for specific use-cases.
This talk is accessible to all, but will be particularly valuable for forecasting practitioners and those interested in the intersection of time series analysis and generative AI.
Outline
Minutes 1-5: Introduction to transformer-based forecasters and TSFMs
- Use-cases where they excel
- How TSFMs differ from traditional forecasting models
Minutes 5-10: The data challenge in training TSFMs
- Survey of available open datasets
- The role and potential of synthetic data
Minutes 10-25: Analysis of state-of-the-art TSFMs
- Overview of Chronos (AWS) and TimesFM (Google)
- Architectural choices and innovations
- Discretization strategies for continuous variables
- Data mixing approaches
- Key findings from ablation studies
Minutes 25-30: Conclusion and future directions
- Research gaps and opportunities
- Guidelines for conducting your own benchmarks
- Q&A
No previous knowledge expected
Machine Learning Engineer by day and Open Source maintainer by night, Luca is passionate about time-series. Feel free to reach out to me on LinkedIn for feedback and/or material! https://www.linkedin.com/in/lucabaggi/