Evolving Netflix's Ray Platform for the GenAI Era. Highlight Talk

The generative AI revolution has transformed the world of large-scale deep learning infrastructure. Modern machine learning platforms must be ready to support pre-training for massive foundation models, memory-intensive fine-tuning for LLMs and diffusion models, as well as low-latency deployments for multi-billion-parameter models.

Navigating this emerging landscape requires new techniques and methodologies, leavened with a thorough understanding of the still-nascent GenAI tooling ecosystem. In this talk, we’ll walk through how we’ve adapted and extended Netflix’s production Ray platform to deal with these new challenges

We’ll outline our experiences in deploying Ray for large-scale data processing & curation, LLM fine-tuning, foundation model training, and distributed inference. We’ll also share our strategies for creating GenAI-ready ML infrastructure to support the needs of the modern deep learning landscape.

Ray Summit 2024 Highlight

Session https://raysummit.anyscale.com/flow/anyscale/raysummit2024/landing/page/sessioncatalog/session/1722028596844001bCg0

Slides for the talk can be found here