Senior Staff Applied AI Inference Engineer

Join our team as a Senior Staff Applied AI Inference Engineer, where you will focus on optimizing large language models for production. You will own the inference stack end to end, working on profiling, optimization, and serving code. This hands-on engineering role also involves customer interaction and elements of product and technical solutions work. You will design and optimize serving architectures, adapt and scale optimization methods, and tailor deployments to each customer's needs.

• Profiling and optimizing the inference stack for large language models, including identifying performance bottlenecks and implementing modern optimization techniques.
• Collaborating with customer engineering teams to tailor deployments to their specific needs, ensuring that performance gains are realized in real-world applications.
• Taking ownership of the entire delivery process, from initial experimentation to production deployment, while maintaining a focus on performance goals and clear specifications.

• Hands-on experience shipping code in production with one or more general-purpose languages, such as Python or C++, with a strong preference for Python
• Comfort with modern LLM serving frameworks such as vLLM or SGLang, and with profiling and analyzing performance down to the kernel level
• Clear interest and hands-on experience with large language models
• A working knowledge of AI/ML pipelines and the full path of developing and deploying ML models
• Familiarity with methods for optimizing LLMs for high throughput / low latency inference
• A firm grasp of how GPUs are built and how they behave
• A Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field
• Strong communication skills, particularly when explaining hard technical topics to customers and teammates
• A track record of making software systems run faster, especially for large language models
• Experience with CUDA or comparable technologies
• A strong command of software engineering fundamentals, with a record of building and shipping AI/ML inference systems
• Experience with Docker and Kubernetes
• Prior work building or tuning AI/ML projects, particularly in a customer-facing setting