Senior ML Infrastructure Engineer

Join SambaNova, an inference-first company, as a Senior ML Infrastructure Engineer. In this role, you will design, build, and operate the production-grade inference infrastructure that powers SambaNova's serving stack on our Reconfigurable Dataflow Unit (RDU) architecture. You will work closely with ML, compiler, runtime, and product teams to ship inference features from prototype to production. Key responsibilities include optimizing advanced inference techniques for performance and cost, owning the public inference API surface, and building and maintaining the accuracy verification and regression infrastructure.

• Design, build, and operate the production-grade inference infrastructure that powers SambaNova's serving stack on RDU architecture.
• Own end-to-end systems spanning request scheduling, advanced decoding algorithms, caching layers, API surfaces, and the accuracy infrastructure.
• Partner closely with ML, compiler, runtime, and product teams to ship inference features from prototype to production.

• Experience designing and maintaining production services with strict latency, throughput, and availability requirements
• Working knowledge of modern LLM inference techniques and familiarity with open-source serving stacks such as vLLM, TensorRT-LLM, or SGLang
• Experience collaborating across teams to deliver complex, system-level engineering solutions
• 5+ years of industry experience building and operating large-scale distributed systems, ideally in ML serving
• B.S. in Computer Science, Electrical Engineering, or related field
• Strong software engineering fundamentals: algorithms, data structures, concurrency, and systems design
• Proficiency in Python