Allison Wang
Allison Wang is a Software Engineer at Databricks and an Apache Spark Committer, specializing in Spark SQL and PySpark. She’s passionate about bridging Python with the big data ecosystem. Allison holds a bachelor’s degree in Computer Science from Carnegie Mellon University.
Sessions
12-04
14:00
30min
Bridging Big Data and AI: Empowering PySpark with Lance Format for Multi-Modal AI Data Pipelines
LU QIU, Allison Wang
By unifying PySpark's robust big data processing/analyzing capability with Lance's multimodal AI data lake, data engineers and scientists can efficiently manage and analyze the diverse data types required for cutting-edge AI applications within a familiar big data framework.
AI/ML Track
AI/ML Track