Watch Netflix movies & TV shows online or stream right to your smart TV, game console, PC, Mac, mobile, tablet and more. Backed by IVP and Redpoint.
About the role
Netflix is building toward more relevant, more varied, and more trustworthy recommendations, and thoughtful, high-quality evaluation is essential to making sure we are moving in the right direction. Join a team creating the frameworks, tools, and workflows that ensure human judgment is applied with consistency, clarity, and care, whether we are evaluating relevance, repetition, diversity, or whether a recommendation simply makes sense to the member. Found on 1752vc Careers, the job board for startup and VC roles.
What they're looking for
- Lead end-to-end execution of human evaluation and data operations initiatives, from intake and scoping to delivery
- Partner with Product Management to translate recommendation strategy and roadmap priorities into evaluation designs, success metrics, and timelines
- Manage human annotation vendors, including capacity planning, SLAs, quality performance, and escalations
- Drive operational efficiency across evaluation programs, including cost per task, throughput, turnaround time, and speed to IRR
- Develop clear, scalable guidelines and scoring rubrics to ensure consistent rater judgment, including for subjective, taste-based questions
- Oversee rater onboarding, calibration, and QA workflows
More about this role
At Netflix, our mission is to entertain the world. Together, we are writing the next episode - pushing the boundaries of storytelling, global fandom and making the unimaginable a reality. We are a dream team obsessed with the uncomfortable excitement of discovering what happens when you merge creativity, intuition and cutting-edge technology. Come be a part of what’s next.
Great recommendations are how members find something worth their time. We are looking for an experienced Human Evaluations Program Manager to help us measure and improve the quality of recommendations across the Netflix homepage and discovery experience. You will work alongside Product Management and ML teams to build the human evaluation programs that tell us whether recommendations are working for members, and where they are not.
Netflix is building toward more relevant, more varied, and more trustworthy recommendations, and thoughtful, high-quality evaluation is essential to making sure we are moving in the right direction. Join a team creating the frameworks, tools, and workflows that ensure human judgment is applied with consistency, clarity, and care, whether we are evaluating relevance, repetition,...
Browse similar: Startup jobs · Remote jobs