# Lead Machine Learning Engineer, Evaluations at ASAPP

- Company: ASAPP
- What the company does: ASAPP delivers the leading agentic customer experience platform for enterprises at scale.
- Company website: https://www.asapp.com
- Type: Startups (AI role)
- Level: Senior
- Location: Mountain View
- Work setup: Hybrid
- Pay: $170K to $190K base salary per year (USD)
- Posted: 2026-09-11
- Apply by: 2026-10-26
- Apply: https://jobs.lever.co/asapp-2/3ad3aa9f-07d7-4af3-9997-fe3d1ae2f5c8
- Page: https://www.1752.vc/careers/jobs/asapp-lead-machine-learning-engineer-evaluations/

## About the role

Help develop the technical roadmap and architecture for the evaluation platform, from offline benchmarking to online/production monitoring of agentic and LLM-based systems. Design eval methodologies appropriate to different stages of the pipeline: golden/regression test sets, human-in-the-loop review workflows, LLM-as-judge approaches, and automated metrics for task success, safety, and hallucinations.

## What they're looking for

- Deep, hands-on experience building and operating evaluation systems for modern ML/LLM/agentic systems- not just consuming existing eval tools
- Demonstrated experience leading the technical direction of a project or small team: setting architecture, driving design reviews, and being accountable for a system's long-term health (not just shipping features)
- Strong architectural skills, with proven experience designing complex, data-intensive software systems and production experience with Python, AWS, Kubernetes, and/or Docker
- Experience designing data pipelines for ML evaluation- labeling/annotation workflows, dataset versioning and quality control, and reproducible benchmarking
- A Bachelor’s Degree in CS or other related fields
- Demonstrated technical mentorship of junior and mid-level engineers, driving adoption of best practices and architectural alignment for scalability and extensibility

Tags: Engineering