Make intelligence open and accessible to all. Backed by Battery, Lightspeed and Sequoia.
About the role
You will build the evaluation infrastructure that researchers use to understand model capabilities, guide training decisions, and assess readiness for release. The role spans evaluation authoring, distributed execution, and results analysis, with a particular focus on pre-training. You will work directly with researchers to make evaluations faster, more reliable, and easier to use, ensuring that the results accurately reflect the models being evaluated. Found on 1752vc Careers, the job board for startup and VC roles.
More about this role
Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.
You will build the evaluation infrastructure that researchers use to understand model capabilities, guide training decisions, and assess readiness for release. The role spans evaluation authoring, distributed execution, and results analysis, with a particular focus on pre-training. You will work directly with researchers to make evaluations faster, more reliable, and easier to use, ensuring that the results accurately reflect the models being evaluated.
Design and evolve evaluation abstractions. Build clear APIs and data models for evaluation tasks, datasets, configurations, model outputs, and scores, making it straightforward to add new evaluations and extend existing workflows.
Build scalable, reliable execution infrastructure. Develop distributed systems for evaluating model checkpoints throughout training and ahead of release, with robust orchestration, observability, and recovery from failures.
Optimize...
Browse similar: AI jobs · AI startup jobs · Startup jobs · New York