# AI Evaluation Lead at Elly

- Company: Elly
- What the company does: The first empathetic audio companion for patients living with a chronic disease. Backed by Techstars.
- Company website: https://ellyhealth.com
- Type: Startups (AI role)
- Level: Senior
- Location: United States
- Work setup: On-site
- Posted: 2026-07-17
- Apply by: 2026-10-08
- Apply: https://job-boards.greenhouse.io/elly/jobs/4323525009
- Page: https://www.1752.vc/careers/jobs/elly-ai-evaluation-lead/

## About the role

Our client is hiring an AI Evaluation Lead to own how we measure the quality of AI-generated financial advice. Getting the advice right matters. A bad output here has real consequences for real people, and this role owns making sure we catch it.

## What they're looking for

- Fluency with how LLM-based systems behave in production, including output variance, failure modes, and the limits of automated scoring
- Ability to assess whether an eval framework is measuring the right things, not just whether it is running correctly
- Comfortable working with behavioral and interaction data to surface patterns and quality signals
- Familiarity with evaluation and observability tooling

Tags: Engineering