Staff Applied AI Inference Engineer

Join our team as a Staff Applied AI Inference Engineer, where you will focus on optimizing large language models for production use. You will own the inference stack end to end, working on profiling, optimization, and serving code. This hands-on engineering role will involve coding, profiling, and low-level optimization, as well as direct collaboration with customer engineering teams. You will have the opportunity to make a real impact on customer deployments and contribute to the advancement of AI technology.

• Profiling and optimizing the inference stack for large language models to improve performance and reduce costs.
• Collaborating with customer engineering teams to tailor deployments to their specific needs and ensure successful implementation.
• Taking ownership of the entire delivery process, from initial experimentation to production deployment, while maintaining a focus on performance goals.

• Hands-on experience shipping code in production with one or more general-purpose languages, such as Python or C++, with a strong preference for Python
• Familiarity with methods for optimizing LLMs for high throughput / low latency inference
• Comfort with modern LLM serving frameworks such as vLLM or SGLang, and with profiling and analyzing performance down to the kernel level
• A firm grasp of how GPUs are built and how they behave
• Strong communication skills, particularly when explaining hard technical topics to customers and teammates
• A working knowledge of AI/ML pipelines and the full path of developing and deploying ML models
• Clear interest and hands-on experience with large language models
• A Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field
• A track record of making software systems run faster, especially for large language models
• Experience with CUDA or comparable technologies
• A strong command of software engineering fundamentals, with a record of building and shipping AI/ML inference systems
• Experience with Docker and Kubernetes
• Prior work building or tuning AI/ML projects, particularly in a customer-facing setting