Staff Site Reliability Engineer

Join our team as a Staff Site Reliability Engineer, where you'll take ownership of managing a large fleet of GPU-backed models. You'll design and build our GPU management and scheduling platform, implement admission control, and develop cloud orchestration systems. This is a senior role for someone with over 10 years of experience in site reliability and software engineering.

• Design and build the GPU management and scheduling platform, including the metrics pipeline and admission control logic.
• Implement autoscaling that adjusts the number of model replicas in response to real-time demand and utilization.
• Architect and operate scalable, fault-tolerant, secure production systems on AWS, GCP, or Azure, and develop infrastructure automation and deployment pipelines.

• Strong software engineering fundamentals — you build orchestration and scheduling systems in Python and/or Go, not just configure off-the-shelf tools
• 10+ years of professional experience across site reliability / DevOps engineering and software engineering
• Strong knowledge of containerization and orchestration (Docker, Kubernetes)
• Experience with monitoring and logging stacks (ELK, Grafana, Datadog, or similar)
• Excellent problem-solving skills and the ability to work both independently and collaboratively
• Computer Science Degree Required from a top CS program
• Experience designing systems that make decisions from operational metrics — collecting signals, interpreting them, and driving control loops such as autoscaling, load shedding, or admission control
• Hands-on production experience with at least one major cloud platform (AWS, GCP, or Azure)
• Deep experience with infrastructure automation and CI/CD (Terraform, GitLab CI/CD, or similar)
• Familiarity with secrets management and security tooling (HashiCorp Vault, AWS KMS, Azure Key Vault)
• Strong communication and interpersonal skills
• Experience managing GPU fleets or scheduling workloads across heterogeneous accelerators
• Familiarity with ML inference serving and model deployment (e.g. Triton, KServe, Ray Serve, or similar)
• Experience implementing HIPAA and SOC 2 compliance
• Experience with Kubernetes autoscaling internals (HPA/VPA, custom metrics, custom controllers)
• Experience operating in an HPC environment
• Bachelor's or Master's in Computer Science, Computer Engineering, or a related field