LU QIU
Lu is a Database engineer at LanceDB. Lu builds distributed vector databases at LanceDB and integrates Lance with the big data ecosystem (Spark, Trino). She developed the distributed system Alluxio as its PMC maintainer. She's also a Data on Kubernetes Ambassador and Kubernetes community evangelist, bridging AI data infrastructure with cloud-native technologies.
Sessions
12-04
14:00
30min
Bridging Big Data and AI: Empowering PySpark with Lance Format for Multi-Modal AI Data Pipelines
LU QIU, Allison Wang
By unifying PySpark's robust big data processing/analyzing capability with Lance's multimodal AI data lake, data engineers and scientists can efficiently manage and analyze the diverse data types required for cutting-edge AI applications within a familiar big data framework.
AI/ML Track
AI/ML Track