Armature helps your product get picked by coding agents such as Claude Code, Codex and Cursor. Then it shows you what users do with your product through AI, and runs evals so it keeps working. Backed by Y Combinator.
About the role
You own two things at Armature: the research we publish and the evals system behind our results. You work on both hand in hand with our forward deployed engineer and our product engineer. 1. The research we publish. Our studies on how coding agents think and choose are how most customers find us. You decide what we study next, you run the studies behind our public leaderboards, and you publish the findings under your name. Found on 1752vc Careers, the job board for startup and VC roles.
What they're looking for
- Design and run studies on how coding agents choose tools, how models and harnesses differ, and how agents use software
- Find the next questions worth asking, and new ways to look at the data we already have
- Automate the research runs and the analysis, so each study takes less time than the one before
- Write up your findings so developers want to read and share them
- Test the latest research on evals against our own data, and keep what improves our results
- Turn the questions our customers ask into better evals
More about this role
Language: English is required. You do not need to speak French.
You own two things at Armature: the research we publish and the evals system behind our results. You work on both hand in hand with our forward deployed engineer and our product engineer.
1. The research we publish. Our studies on how coding agents think and choose are how most customers find us. You decide what we study next, you run the studies behind our public leaderboards, and you publish the findings under your name.
2. The evals system. Everything we tell a customer depends on our evals, so they have to be excellent. You research how to design them to get the best results, and you build what works into the product.
This field is young, so many of your questions won’t have an answer yet, and what you find will shape how software gets rebuilt for agents.
- Design and run studies on how coding agents choose tools, how models and harnesses differ, and how agents use software.
- Find the next questions worth asking, and new ways to look at the data we already have.
- Automate the research runs and the analysis, so each study takes less time than the one before.
- Write up your findings so developers want to read and...
Browse similar: AI jobs · AI startup jobs · Startup jobs · Founding team roles · Remote jobs