# Research Manager, Evaluation at Aaru

- Company: Aaru
- What the company does: Humans are unreliable narrators of their own behavior. Memory fails, incentives distort answers, and social pressure warps what people say away from what they actually do. Backed by General Catalyst, Felicis and Redpoint.
- Company website: https://aaru.com/
- Type: Startups
- Level: Senior
- Location: NYC
- Work setup: On-site
- Pay: $280K to $425K base salary per year (USD)
- Posted: 2026-08-22
- Apply by: 2026-10-08
- Apply: https://jobs.ashbyhq.com/aaru/d2fb850d-6d25-4efd-a125-6c2c722deab4
- Page: https://www.1752.vc/careers/jobs/aaru-research-manager-evaluation/

## About the role

As Evaluation Research Manager, you will lead a focused team of Evaluation Researchers and research engineers. You will translate Aaru's evaluation charter into a coherent portfolio of studies, datasets, and shared infrastructure, and you will be accountable for the quality, pace, and usefulness of the team's work.

## What they're looking for

- Two teams disagree about whether a system improved because they use different metrics and test sets. Identify the underlying construct, choose the right evidence, and create a shared evaluation that resolves the disagreement
- A population looks plausible one profile at a time but fails to reproduce important real-world relationships. Build measurements that expose the gap and help Population Research identify its cause
- A prediction method is well calibrated overall but systematically overconfident for a high-value subgroup. Determine whether the issue is data coverage, model structure, selection, condition shift, or the evaluation itself
- An offline benchmark has become a development target and is beginning to leak into decisions. Redesign the evaluation system so teams can iterate quickly without exhausting the integrity of the final holdout
- A component metric improves, but customer decisions do not. Determine whether the metric is invalid, the effect is too small, downstream components erase the gain, or the product is presenting the result incorrectly
- Ground truth is delayed, noisy, incomplete, or open to multiple interpretations. Design a study that remains useful without pretending that the label is cleaner than it is

Tags: Technical Staff