# Member of Technical Staff (Evals) at Jarmin

- Company: Jarmin (YC F25)
- What the company does: AI Employees for human-level roles. Backed by Y Combinator.
- Company website: https://jarmin.ai/
- Type: Startups (AI role)
- Level: Senior
- Location: San Francisco, CA, US
- Work setup: On-site
- Pay: $200K to $300K base salary per year (USD)
- Posted: 2026-08-17
- Apply by: 2026-10-08
- Apply: https://www.ycombinator.com/companies/jarmin/jobs/K9ot5wF-member-of-technical-staff-evals
- Page: https://www.1752.vc/careers/jobs/jarmin-member-of-technical-staff-evals/

## About the role

As a Member of Technical Staff on Evals, you'll own how we measure AI Employee performance on the job.

## What they're looking for

- Evaluation systems: datasets, offline replay, scorers, and regression alerts
- Feedback loops for models and agents
- Autonomous failure analysis
- Defining what good, degraded, and failed runs look like, and alerting on them
- Have built evaluation or measurement systems, such as AI evals, experimentation, ranking, or search quality
- Can turn ambiguous quality questions into concrete metrics and decisions

Tags: Engineering