Sean Law - Modern Time Series Analysis with STUMPY - Intro To Matrix Profiles | PyData Global 2020

Traditional time series analysis techniques have found success in a variety of data mining tasks. However, they often require years of experience to master and the recent development of straightforward, easy-to-use analysis tools has been lacking. STUMPY is a scientific Python library for modern time series analysis that efficiently computes something called a matrix profile and leverages popular open source software and enables you to do better science!
github.com/TDAmeritrade/stumpy
stumpy.readthedocs.io
Sean Law is a senior applied scientific researcher and lead data scientist currently working with a multi-talented Exploration Lab team and serves as an advisor on an enterprise A.I. Council at TD Ameritrade. He has experience producing cutting edge methodologies, building high-performance predictive models, and developing rapid prototypes. Additionally, he is one of the co-organizers of PyData Ann Arbor and is also the creator and core maintainer of STUMPY, a powerful and scalable open source Python library that can be used for a variety of time series data mining tasks.
www.pydata.org
PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.
PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases. 00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.
Want to help add timestamps to our KZbin videos to help with discoverability? Find out more here: github.com/numfocus/KZbinVi...

[@lu.quit.a]

This was awesome, very good introduction to Matrix Profiles!

[@jefferyanderson]

Excellent presentation Sean

[@stephanembatchou5300]

That was a great presentation.

[@MrStephcaster]

awesome! I can't wait to test it

[@SreeramAjay]

Very nice explanation, thank you

[@abolfazlzeraatkar2425]

very good and really ease to learn, thanks

[@TerritOrially]

🎯 Key Takeaways for quick navigation: 00:04 *🌐 Introduction to STUMPY and Modern Time Series Analysis* - Introduction by Sean Law, the creator and core developer of STUMPY, highlighting the agenda of the presentation on modern time series analysis using STUMPY. - Sean Law thanks the PyData Global organizers and TD Ameritrade, his employer. - Challenges in time series data analysis are outlined, emphasizing the difficulty in visualizing and analyzing large datasets. 02:08 *📊 Common Approaches in Time Series Analysis* - Overview of traditional methods in time series analysis, including visualization, statistics, auto-regressive models, anomaly detection, forecasting, machine learning models, unsupervised learning, dynamic time warping, and deep learning. - Each method's limitations and the necessity for a scalable and intuitive solution are discussed. 04:01 *📘 Illustrative Example of Time Series Analysis* - Introduction to basic concepts in time series analysis, such as subsequences, and the goal of identifying conserved behaviors or patterns. - The presentation outlines the importance of a simple and intuitive approach that is easy to interpret, user and data agnostic, requires no prior knowledge, and is (nearly) parameter-free. 06:50 *🔍 Comparing Subsequences Using Euclidean Distance* - Explanation of how to compare subsequences in a time series using Euclidean distance. - The process of computing a distance matrix for time series data is detailed, illustrating the computational challenge it presents for large datasets. 09:35 *🧮 Introduction to the Matrix Profile Concept* - Introduction to the matrix profile concept, a transformative approach that simplifies the analysis of time series data by focusing on the nearest neighbors of subsequences. - The matrix profile's ability to identify motifs (repeated patterns) and discords (anomalies) in time series data is highlighted. 13:47 *📈 Using the Matrix Profile for Analysis* - Practical applications of the matrix profile in identifying conserved behaviors, motifs, and potential anomalies within time series data. - The significance of the matrix profile index and how it aids in locating the nearest neighbors for subsequences is discussed. 16:34 *💻 Development and Impact of STUMPY* - The evolution of algorithms to efficiently compute the matrix profile, highlighting the contributions of the STAMP, STOMP, and GPU-accelerated STOMP algorithms. - Introduction of STUMPY, a Python library developed to validate and implement matrix profile algorithms for efficient time series analysis. 17:59 *🚀 STUMPY: A Powerful Tool for Time Series Analysis* - STUMPY is highlighted as a scalable Python library for computing the matrix profile, enabling efficient time series analysis without reinventing the wheel. - STUMPY's growth and community support: over 50,000 downloads, 1,500 GitHub stars, and current version 1.5. - Features and capabilities: minimal dependencies, compatibility with modern Python, parallelization across server cores, Dask cluster support for distribution across multiple servers, and performance benchmarks (256 CPUs across 32 servers computed a 100 million data point sample matrix profile in under 10 days). 21:20 *📊 Live Demo: Analyzing Time Series with STUMPY* - A live demonstration showcases STUMPY's ability to identify patterns and anomalies in time series data through the computation of matrix profiles. - Detailed examination of a specific pattern within the time series, its repetition, and the process of identifying its nearest neighbor. - The demo highlights the matrix profile's role in detecting motifs (conserved patterns or behaviors) and discords (potential anomalies) within the time series, illustrating how these insights guide where to focus analysis. 26:39 *📘 STUMPY Documentation and Resources* - Overview of STUMPY's extensive documentation, tutorials, and API, which support users in efficiently utilizing the library for time series analysis. - The versatility of STUMPY is showcased, including its application to multi-dimensional matrix profiles, time series chains, semantic segmentation, and the comparison and clustering of time series data. - Encouragement for community contribution, communication about STUMPY, and utilization of the library, alongside links to tutorials, live demos, and open-source code repositories. Made with HARPA AI

[@ranii3116]

Good work. But how can it be combined with other methods (clustering,ml, arima etc.) I thought this is indivisual method that parallel to those methods.

[@yeongnamtan]

that was a very interesting presentation. Would you be able to share your code so we could play around with stumpy ?

[@gianmarcosalvi339]

Is it possible to apply AB join to timeseries with different granularity data? i.e. T_a has data sampled every 200ms and T_b has data sampled every 10s

[@21jvg2]

I think you can try to generate some noice between, or use dynamic warping to match

[@RyuuOujiXS]

subsequence in series is how you can tell someone is stupid. It should be subseries.