Firecrawl is the web data API to search, scrape, and interact with the web at scale. Turn any source into clean Markdown or structured data your agents can ship with. Backed by Y Combinator.
About the role
Design the metrics that define what "good output" actually means across millions of sites, formats, and edge cases Build the pipelines and harnesses that measure quality rigorously and at scale
What they're looking for
- You have the engineering depth to build real evaluation systems, not just run existing ones
- You care deeply about what "good" means and how to measure it rigorously
- You're comfortable owning ambiguous problems where the metric itself has to be invented
- You move fast and close the loop. You'd rather ship, measure, and iterate than perfect on paper
- Someone who only wants to run benchmarks someone else designed
- A pure researcher who won't build the systems, or a pure engineer who won't think about methodology
More about this role
You'll build the evaluation systems that tell us whether Firecrawl actually works. That sounds simple. It isn't. Our core promise, convert any URL into clean, structured, LLM-ready data reliably, is hard to measure rigorously across millions of different websites, formats, and edge cases. As the systems we're measuring get more complex, the question "did that work?" gets harder, not easier.
This isn't an eval role where you inherit a framework and run benchmarks. You'll design the metrics, build the pipelines, generate the datasets, and own the feedback loop from output quality back to model and product decisions. If you care about what "good" actually means and have the engineering depth to measure it, this is the role.
Equity Range: Competitive equity. Details shared during the process.
Job Type: Full-Time
Experience: 4+ years in ML, research engineering, or data-heavy backend, with real evaluation work
Work Authorization: Must be authorized to work in the United States or Canada. We're not able to sponsor US visas right now. For Canada, we'll consider sponsorship on a case-by-case basis through our Toronto Hub.
Firecrawl is the easiest way to turn the web into data AI agents...
Browse similar: AI jobs · AI startup jobs · Startup jobs · San Francisco Bay Area