Member of Technical Staff (Evals)

Join Handshake, a rapidly growing company at the forefront of AI research and development. As a Member of Technical Staff (Evals), you will play a crucial role in defining how frontier AI systems are measured, understood, and improved. You will work closely with AI researchers, domain experts, and customers to develop new benchmarks, reward and verifier systems, agent-evaluation methodologies, and data-quality techniques. This is a high-ownership role that offers the opportunity to shape the technical direction, operating culture, and research products of the company.

• Definir cómo se miden, comprenden y mejoran los sistemas de IA de vanguardia, trabajando en preguntas clave sobre qué se debe medir y cómo diseñar evaluaciones.
• Colaborar con investigadores de IA, expertos en dominios y clientes para desarrollar nuevos estándares, sistemas de recompensa y metodologías de evaluación.
• Construir sistemas para datos humanos de alta calidad, incluyendo el diseño de tareas, metodologías de anotación y señales de calidad de datos.

• Comfort operating in an ambiguous, fast-moving environment with substantial ownership
• PhD in ML/AI, computer science, data science, or related fields (or equivalent research experience in industry)
• Publications at top AI/ML venues like NeurIPS, ICML, ICLR, COLM
• Strong Python skills, experience building scalable software, working with agents
• Builders who enjoy tinkering with agents and shipping high-quality software, benchmarks, or datasets (e.g. a strong GitHub profile / OSS contributions, or product portfolio)
• Strong knowledge of frontier AI: benchmarks, eval techniques, agent harnesses, post-training recipes, data shapes
• Join a rapidly growing company whose data business grew from zero to nearly $1B run rate in a year
• Help build an early technical organization where your work shapes the roadmap, standards, and culture
• Work at the very frontier of AI with most major AI labs, researching some of the most important problems in Data and Evaluations
• Attend (and publish at) conferences like NeurIPS, ICML, ICLR, COLM
• Publish results and work in public through open-source benchmarks and software, papers, and blogs