From the first symptom to full recovery, Nolla guides you to the care you need now and helps you plan for the future. Every diagnosis and prescription is reviewed and approved by a licensed U.S. medical provider. Backed by General Catalyst.
About the role
Build harnesses that run agents through reproducible tasks, capture their actions, and make failures easy to investigate. Develop multi-step and multi-visit environments with patient state, clinical tools, and realistic consequences for an agent’s decisions. Found on 1752vc Careers, the job board for startup and VC roles.
What they're looking for
- Strong programming skills and experience building reliable software for experiments, data processing, or agent systems
- Hands-on experience building an agent evaluation harness, an RL environment, or closely related infrastructure
- The ability to translate an open-ended problem into a runnable task with measurable outcomes
- Careful judgment about data quality, reproducibility, grading reliability, and what a benchmark result actually supports
- Comfort debugging across datasets, environment state, agent behavior, and scoring code
- The ability to work with clinicians and turn domain expertise into precise specifications and tests
More about this role
New York City · Full-time
At Nolla Health, we’re building AI-powered healthcare. To improve the systems involved in delivering care, we need realistic ways to test them: cases that unfold over time, tools an agent can use, and clear criteria for judging what happens.
We’re looking for a research engineer to build evaluation harnesses, simulated environments, and data pipelines that turn clinical knowledge and appropriately governed data into useful evaluation and training assets.
You’ll work with engineers and clinicians to define tasks, build the systems that run them, and make the results trustworthy. The work will support our own agents and models, with the opportunity to develop benchmarks and environments for external AI teams.
Build harnesses that run agents through reproducible tasks, capture their actions, and make failures easy to investigate.
Develop multi-step and multi-visit environments with patient state, clinical tools, and realistic consequences for an agent’s decisions.
Work with clinicians to turn clinical workflows into well-defined tasks, rubrics, and success criteria.
Build data pipelines for preparing, validating, versioning, and tracing evaluation and...
Browse similar: Startup jobs