12-04, 13:30–15:00 (UTC), LLM Track
Colab Notebook Link: https://colab.research.google.com/drive/1faxDHE3LdAwH7MORdnJei87Q0WF1BhS0?usp=sharing
Make a copy to your local drive to start working on this notebook.
Ever wondered how groundbreaking language models like ChatGPT and Llama were built? The answer lies in transformer, a powerful neural network architecture. In this workshop, we'll dive deep into the inner workings of transformers, with specific focus on self-attention mechanism. We will guide you through the process of building one from scratch. Whether you're a beginner or an experienced practitioner, this workshop is designed to cater to all levels of expertise.
Target Audience: This beginner-level workshop is designed for students, data enthusiasts, data scientists, or machine learning practitioners with foundational knowledge of Python and NumPy.
Workshop Structure:
-
Introduction to transformer (10 minutes)
-
Notebook implementation of self-attention (1 hour)
-
Summary, discussion and Q&A (20 minutes)
No previous knowledge expected
Shefali is completing her MS in Applied Statistics at Columbia University and brings over 3 years of data science and analytics experience across education, consulting, and AdTech. Her notable work includes leading data-driven initiatives that impacted 90,000 schools at BCG, optimizing ad performance at Media.net (one of the top 5 largest AdTech companies worldwide by market cap), and developing cloud-based analytics solutions for U.S. non-profit institutions.
I am a quantitative modeling senior associate at JPMorgan and I hold a PhD in Economics.
Feel free to connect me on LinkedIn: https://www.linkedin.com/in/chuxin-liu/
Sheetal is a Senior Applied Scientist at Etsy and has six years of experience in data science and machine learning, with a career that spans Asia and Europe and active engagement with the global data science community. She worked at Amazon as an Applied Scientist in London, focusing on personalization, and as a Machine Learning Engineer at JP Morgan Chase in Hong Kong. She holds a master’s degree in Data Science and AI from a dual degree program in the Netherlands and Finland, during which she published papers at top-tier conferences. Currently, she leads a paper reading group in the Northeast, facilitating discussions with fellow data professionals.
Sheetal is deeply passionate about fostering and growing women-focused communities in tech. As a WiDS (Women in Data Science) ambassador, she actively supports initiatives that empower women in the field. Her dedication to community impact was recognized with the Social Impact Award in Germany.