MAPS: Netflix’s Multimodal Asset Personalization at Scale
Netflix describes how multimodal embeddings from CLIP and MediaFM improved asset personalization by solving cold-start problems, consolidating artwork models across canvases, enabling query-aware search artwork, and boosting video preview ranking, all supported by a shared Embedding Store and validated through offline IPS and online A/B tests.
MAIN POINTS
- CLIP image embeddings let artwork models personalize new assets immediately, reducing cold-start dependence on interaction history.
- A unified model across five artwork canvases outperformed separate per-canvas models, especially on sparse canvases.
- MediaFM’s visual, audio, and text embeddings beat ID-only and SeqCLIP for video preview personalization.
- A linear probe cheaply screened candidate embeddings before expensive offline evaluation and online experiments.
TAKEAWAYS
- Content-aware embeddings transfer preference signals across titles and formats better than opaque asset IDs.
- Multimodal representations matter most when visual-only signals miss important cues like dialogue and music.
- Shared embedding infrastructure makes foundation-model upgrades reusable across multiple personalization systems.
- Cheap proxy tasks can significantly reduce experimentation cost while still predicting which embeddings will perform best.