Preference Model is building the next generation of training data to power the future of AI. Backed by a16z and South Park Commons.
About the role
An RL environment is only useful once it runs reliably inside a lab's training stack. Every environment we build has to be packaged, sandboxed, run at scale, checked against each lab's requirements, and delivered in the format that lab's platform expects. Much of that work is still manual today, and it takes time away from the people designing environments. Found on 1752vc Careers, the job board for startup and VC roles.
What they're looking for
- 5+ years of experience building and owning ML Platform Engineering, ideally AI related ML Platform Engineering
- Strong software engineering fundamentals and a track record of building and operating production infrastructure or developer platforms that other engineers depend on
- Hands-on experience with containers and orchestration (Docker, Kubernetes, or similar) and at least one major cloud platform (GCP, AWS, or similar)
- A working understanding of how to run untrusted code safely: isolation, resource limits, and what happens when a process fills the disk or refuses to exit
- You've owned something tedious end to end, such as a migration, a release process, or a delivery pipeline, and automated away as much of it as you could
- Extensive hands-on use of coding agents (Claude Code, Cursor, Codex, or similar), and an intuition for where models cut corners
More about this role
Preference Model is a superintelligence data research company. We build RL environments for training capable, better-aligned superintelligences.
Almost every aspect of how a model behaves is shaped by the reward signals it's trained on. The biggest problem with AIs today is that they don't always do what we intend. Sometimes that's because they aren't capable enough, and sometimes it's because they aren't aligned. Both problems come down to the training objectives.
We're trying to address both of these issues in the most direct, highest leverage way available to us, which today means researching how to build RL environments better, not just for the AIs of today but for the AIs as they will be when they outsmart us.
Over the past year, we've built RL environments for several frontier labs, and we're backed by $16M in seed funding led by a16z. Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude.
An RL environment is only useful once it runs reliably inside a lab's training stack. Every environment we build has to be packaged, sandboxed, run at scale, checked against each lab's requirements, and delivered in the...
Browse similar: AI jobs · AI startup jobs · Startup jobs · San Francisco Bay Area