https://pablodelgado.org/tags/dataset-curation/Recent content in Dataset Curation on Share what you knowHugoen-usFri, 21 Nov 2025 12:00:00 +0000https://pablodelgado.org/blog/2025/11/21/how-netflix-builds-machine-learning-datasets-at-scale/Fri, 21 Nov 2025 12:00:00 +0000https://pablodelgado.org/blog/2025/11/21/how-netflix-builds-machine-learning-datasets-at-scale/<p>Multi Modal datasets construction and curation at scale has been a challenging task until recently. We will talk about how Netflix uses Ray to build massive multimodal datasets for text-to-image research. We’ll show how Ray’s distributed processing fans out data ingestion and filtering across hundreds of GPUs, how we run batch inference at scale with cutting-edge vision-language models to score and caption images / videos, and how smart curation and sampling help reduce the size and increase the diversity of datasets producing high quality training data.</p>https://pablodelgado.org/blog/2025/11/05/scaling-multimodal-data-curation-with-ray-and-lancedb/Wed, 05 Nov 2025 12:00:00 +0000https://pablodelgado.org/blog/2025/11/05/scaling-multimodal-data-curation-with-ray-and-lancedb/<p>At Ray Summit 2025, Pablo Delgado from Netflix and Lei Xu from LanceDB share how they are transforming the construction and curation of massive multimodal datasets—traditionally a complex and resource-intensive process—into a scalable, efficient, and highly automated pipeline.</p> <p>They explain how Netflix leverages Ray for distributed ingestion, filtering, and large-scale inference across enormous video and image corpora, while LanceDB serves as the high-performance storage and query layer that provides a single source of truth throughout the data curation lifecycle.</p>