LLM Inference Systems Engineer

Join our team as an LLM Inference Systems Engineer, where you will be responsible for building the distributed serving and high-performance networking layer behind our large language models. You will work closely with the Model, Infrastructure, and Application engineering teams to design, implement, and operate disaggregated LLM serving architectures. Your expertise in optimizing multi-node inference across modern GPU systems and diagnosing complex distributed failures will be crucial to the success of this role. A Master's or PhD is required, along with experience in deploying production workloads on Kubernetes and a strong understanding of LLM inference.

• Conception, mise en œuvre et exploitation d'architectures de service LLM désagrégées, y compris les pools de pré-remplissage et de décodage indépendants.
• Optimisation de l'inférence multi-nœuds sur des systèmes GPU modernes en utilisant NVLink, NVSwitch, NVLS, InfiniBand, RoCEv2, GPUDirect RDMA, NCCL, UCX.
• Production de services sur Kubernetes avec des vérifications de santé, une mise à l'échelle automatique, des déploiements sûrs, des métriques, un traçage et des diagnostics exploitables.

• Clear written and verbal communication across research, infrastructure, and product-facing teams
• Master's or PhD Required
• Experience deploying production workloads on Kubernetes and operating them with clear reliability and observability standards
• Strong Python skills and working proficiency in C++ or another systems language
• Ability to profile and debug performance across application, runtime, kernel, network, and infrastructure layers
• Strong understanding of LLM inference, including tensor/pipeline/data parallelism, continuous batching, KV-cache management, and prefill/decode behavior
• Production experience building or operating distributed systems, high-performance computing systems, or large-scale ML inference platforms
• Hands-on experience with GPU communication and networking technologies such as NCCL, NVLink/NVSwitch, InfiniBand, RoCE, RDMA, UCX, or equivalent systems
• Direct experience with disaggregated serving, KV-cache transfer, NVIDIA Dynamo/NIXL, SGLang, vLLM, or TensorRT-LLM
• CUDA, Triton, custom kernel, or low-level GPU performance experience
• Experience with NVLS, SHARP, GPUDirect RDMA, UCX, RDMA congestion control, or GPU-cluster topology optimization
• Experience with speculative decoding, multi-LoRA serving, quantization, or cache-aware routing
• Contributions to open-source inference, networking, or distributed-systems projects
• Experience benchmarking new GPU platforms and turning results into production architecture decisions