Staff Software Engineer (RL Environments)

Join Scale as a Staff Software Engineer, where you'll design and build RL environments at scale. This hands-on engineering role involves setting technical direction across multiple teams and writing the hard part of the code. You'll work on both platform design and environment development, making a significant impact on the future of AI.

• Conception et développement d'une plateforme pour l'exécution, l'emballage et la version des environnements RL.
• Conception et mise en œuvre de systèmes pour orchestrer les déploiements, capturer les trajectoires et vérifier les résultats.
• Direction technique et leadership dans la résolution de problèmes complexes liés à l'ingénierie logicielle.

• Hands-on experience building with LLMs including agent loops, tool calling, MCP, or eval harnesses, and enough intuition about model behavior to reason about what a training signal actually teaches
• Experience building or operating high-throughput backend systems: orchestration, job scheduling, queuing, and large-scale data pipelines
• Demonstrated ability to own ambiguous, undefined problems end to end and drive them to a shipped system
• Excellent written and verbal communication; ability to align engineers, researchers, and non-engineering partners on a technical direction
• Strong Python skills and a track record of shipping production software; comfort in at least one other part of the stack (TypeScript/React, Go, Rust, or similar)
• 8+ years of software engineering experience with strong fundamentals in distributed systems, system design, data structures, and algorithms
• Deep experience with containerization and sandboxed execution, including Docker, VMs, gVisor/Firecracker, Kubernetes, or equivalent
• Experience with high-scale sandbox or code-execution infrastructure, remote development environments, or CI systems
• Experience with cloud-native infrastructure across AWS/GCP/Azure, Infrastructure as Code, and CI/CD
• Experience with RL training or serving stacks (verl, TRL, Ray, vLLM, SGLang, or similar)
• Familiarity with post-training methods: RLHF, RLAIF, RLVR, GRPO/PPO-family algorithms, rejection sampling, reward modeling, and the practical failure modes of each
• Experience building internal tools that non-engineers rely on daily, especially data-dense review and annotation interfaces
• Experience designing verifiable reward signals, and firsthand experience with reward hacking and how to defend against it
• Strong observability instincts: tracing, structured logging, and metrics for systems whose failure modes are statistical rather than binary
• Direct experience building RL environments, agentic benchmarks, or eval harnesses (SWE-bench-style task suites, terminal or browser environments, tool-use benchmarks, or in-house equivalents)
• Experience in a research-adjacent engineering role, translating research goals into production systems
• Experience working directly with sophisticated external technical customers
• Prior technical leadership at staff level or above in a fast-moving, ambiguous environment