Machine Learning Engineer (Real-Time Speech Translation)

Join LILT, a leading language translation technology company, as a Machine Learning Engineer focused on real-time speech translation. In this hands-on backend engineering role, you will own the real-time speech translation backend from live audio input to translated output. You will work closely with senior architects and language processing researchers, integrating and serving streaming speech recognition and machine translation models. You will also architect and scale production ML infrastructure on GPU-accelerated Kubernetes clusters, establish comprehensive instrumentation for real-time performance, and define technical contracts and interfaces for audio ingestion and downstream service integrations.

• Conception et gestion de services pour le streaming audio et texte en temps réel, en assurant le traitement du signal et la gestion de la concurrence.
• Intégration et service de modèles de reconnaissance vocale et de traduction automatique en streaming, en collaborant avec les équipes de recherche.
• Architecture et mise à l'échelle de l'infrastructure ML de production sur des clusters Kubernetes accélérés par GPU.

• 3+ years building production backend or ML serving systems in Python, including strong async programming (asyncio) skills
• Experience integrating speech or NLP models into production systems, ideally streaming ASR (partial hypotheses, endpointing, VAD)
• US citizenship and residence in the United States (contract requirement)
• BS or MS in Computer Science or a related field, or equivalent practical experience
• Hands-on experience with real-time streaming transport: WebSocket or gRPC bidirectional streaming, session state, backpressure, and connection lifecycle handling
• A latency-engineering mindset: you have profiled, instrumented, and optimized a real-time or low-latency system and can reason in per-stage budgets
• Experience serving ML models in production on GPUs (Ray Serve, Triton, vLLM, or similar), with Docker and Kubernetes
• Effective use of AI coding agents (Claude Code, Codex, or similar) on top of fundamentals learned the hard way: you let agents do the typing, but you can debug, review, and reason about every line without them, and you know when not to trust them
• Observability tooling (Datadog, Prometheus) for production ML systems
• Handling of CJK and other non-Latin text in NLP pipelines (our first languages are Japanese, Korean, and English)
• WebRTC and SFU concepts, or voice pipeline frameworks (LiveKit Agents, Pipecat)
• Streaming text-to-speech integration and time-to-first-audio optimization
• Machine translation quality estimation (COMET/CometKiwi class models) or other confidence estimation in production
• Message brokers for real-time fan-out and state distribution (RabbitMQ or similar)
• Familiarity with simultaneous or incremental MT concepts (retranslation, prefix stability, wait-k policies)
• Ray Serve specifically, including streaming responses and model multiplexing