Software Engineer (Inference Platform)

Join Baseten, a remote-first company, as a Software Engineer focused on building the distributed runtime for large-scale LLM inference. You'll work across the stack, ensuring models are fast, reliable, and cost-efficient. You'll have the opportunity to own projects end-to-end and make a significant impact on how developers use AI models at scale. Benefits include unlimited PTO, full healthcare coverage, and a learning and development budget.

• Construire l'infrastructure et les systèmes d'orchestration qui déploient et exécutent l'inférence LLM distribuée à grande échelle, y compris le routage, l'autoscaling, la planification et la gestion du runtime.
• Concevoir, construire et exploiter les API de modèle, en mettant l'accent sur les capacités d'inférence avancées : sorties structurées, appel d'outils/fonctions et service multimodal.
• Déboguer et renforcer des systèmes de production complexes couvrant Kubernetes, les runtimes distribués, le réseau et les charges de travail GPU pour améliorer la fiabilité et l'évolutivité.

• This role is ideal for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users
• 3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs where reliability, latency, and scale are first-class concerns
• Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent practical experience
• Excellent written communication and collaboration skills, including writing clear design docs and working across functions
• A proven track record of owning low-latency, reliable backend services, including rate limiting, auth, quotas, metering, and migrations
• Eagerness to learn new languages, frameworks, and systems, and a real interest in inference engineering. Prior ML or LLM experience isn't required, though experience with model serving or inference systems is a plus
• Comfort debugging performance and reliability issues across multiple layers of the stack, from application behavior down to runtime and infrastructure internals
• Infrastructure instincts with a feel for performance: profiling, tracing, capacity planning, and SLO management
• A strong sense of developer experience. You think about how systems are used, not just how they work
• Experience with or contributions to LLM inference engines and frameworks such as vLLM, SGLang, TensorRT-LLM, TGI, or Dynamo
• Deep Kubernetes experience, including operators and custom resources, plus familiarity with service meshes or API gateways
• Experience with distributed scheduling, autoscaling, or service orchestration
• Experience operating GPU workloads in production
• A background in developer-facing infrastructure or APIs, or contributions to open-source infrastructure or ML systems
• Familiarity with observability tooling, CI/CD systems, or release automation