Member of Technical Staff (ML Platform)

Join Runway, a cutting-edge AI company, as a Member of Technical Staff (ML Platform). In this role, you will own model evaluation end-to-end, designing and building systems that generate samples at scale, score them with automated metrics and human annotations, and track results across checkpoints and releases. You will work closely with research teams and set the technical direction for evaluations across the company. This is a high-leverage role that directly impacts the quality of our models.

• Concevoir et construire les systèmes qui génèrent des échantillons à grande échelle, les évaluent avec des métriques automatisées et des annotations humaines.
• Définir ce que signifie "meilleur" sur le plan opérationnel - comment nous le mesurons, à quel point nous en sommes confiants, et comment un résultat devient une décision d'expédition ou de non-expédition.
• Travailler directement avec les équipes de recherche sur la vidéo, l'image, l'audio, les agents et la robotique pour comprendre leurs besoins en matière de mesure et construire des solutions adaptées.

• Strong systems thinking and pragmatic approach to production reliability
• Ability to lead a broad technical area: gather requirements from many teams, set direction, make tradeoffs, and drive a roadmap without waiting to be told
• Experience designing data pipelines and storage for large volumes of media or model outputs, with attention to versioning and reproducibility
• Self-starter who can work embedded with research teams and move fast
• 5+ years of experience building ML infrastructure or data platforms in production environments, with at least some of that time spent on evaluation, experimentation, or benchmarking systems
• Strong Python and PyTorch, and hands-on experience running large batch GPU workloads on Kubernetes
• Comfort building internal tools end to end, from the command line to the browser
• Familiarity with experimental statistics: paired comparisons, confidence intervals, multiple-comparison pitfalls, inter-rater agreement
• Humility and open mindedness; at Runway we love to learn from one another
• Familiarity with the full model development lifecycle: data, training, evaluation, serving
• Hands-on work with LLM- or VLM-as-judge pipelines
• Track record of building evaluation suites for generative models
• Experience with online experimentation platforms and connecting offline metrics to product outcomes
• Prior work evaluating agents or robotics policies