Software Engineer (Inference Performance)

Join Baseten, a fast-paced startup focused on optimizing AI workloads. As an Inference Performance Engineer, you'll work across the stack to improve the speed and efficiency of our customers' models. You'll have the opportunity to implement cutting-edge inference techniques, profile and optimize performance, and contribute to open-source inference engines. Enjoy a remote-first work environment, unlimited PTO, full healthcare coverage, and a learning and development budget.

• Contribuer à l'optimisation des performances d'inférence des modèles d'IA, en appliquant des techniques avancées telles que la désagrégation de pré-remplissage/décodage et la gestion du cache KV.
• Profilage et optimisation de l'inférence de bout en bout, de l'overhead de lancement du noyau et de la disposition de la mémoire jusqu'à la planification des demandes.
• Développement et mise en production de techniques d'inférence de pointe, en travaillant en profondeur dans les internals d'exécution.

• This role is ideal for someone who thrives in a fast-paced startup environment and is eager to make significant contributions to the exciting field of LLM inference
• Deep understanding of GPU architecture
• Experience with one or more general-purpose programming languages, such as Python or C++
• Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or related field
• Strong familiarity with ML libraries, especially PyTorch, TensorRT, or TensorRT-LLM
• Familiarity with LLM optimization techniques (e.g., quantization, speculative decoding, continuous batching)
• Demonstrated interest and experience in LLMs
• Proficiency in enhancing the performance of software systems, particularly in the context of large language models (LLMs)
• Contributed to vLLM, SGLang, TensorRT-LLM, or another inference engine
• Worked on large-scale distributed serving: autoscaling, load balancing, multi-region or multi-cloud capacity
• Worked on quantization (FP8/FP4, AWQ, GPTQ) or speculative decoding in production
• Written or optimized GPU kernels (CUDA, Triton, CUTLASS, or similar)
• Deep understanding of software engineering principles and a proven track record of developing and deploying AI/ML inference solutions