Production Monitoring for AI agents. Backed by Y Combinator.
About the role
Lemma is production monitoring for AI agents. We catch the silent failures your observability tools and evals miss (think bad tool calls, lost context, and infinite loops) before your users find them. The hardest part of our product is deciding what counts as a failure. Found on 1752vc Careers, the job board for startup and VC roles.
What they're looking for
- High slope over years of experience. New grads and dropouts welcome
- A track record of shipping things people actually use
- Hands-on with LLMs in production: evals, LLM-as-judge, embeddings, and knowing when a smaller model or no model is the right call
- Real research taste. You can tell a real improvement from noise, even when there's no ground truth to check against
- Bonus: you were the customer once. You ran agents in production and got burned
More about this role
Lemma is production monitoring for AI agents. We catch the silent failures your observability tools and evals miss (think bad tool calls, lost context, and infinite loops) before your users find them.
Agents break silently. They call the wrong tool, forget what the user said three turns ago, and loop until someone pulls the plug. Soon they'll be responsible for the majority of the world’s economic work, and most teams won't even know when they fail.
Making agents reliable is the problem Lemma exists to solve. That means catching the unknown unknowns, the failures nobody thought to write an eval for, and closing the loop end to end so they get fixed, not just flagged. It's the foundation for building agents people can actually trust and the future of self-improving systems.
The hardest part of our product is deciding what counts as a failure.
There is no ground truth here, and no benchmark to climb. Every customer's agent is different, what "wrong" means changes from one to the next, and we have to get it right across production without anyone telling us what to look for. Being confidently wrong often costs us more trust than being right fifty times earns.
This role owns the...
Browse similar: AI jobs · AI startup jobs · Startup jobs · San Francisco Bay Area