Startups · AI

Software Engineer L5 - AI Observability & Agent Evaluation

Netflix · Los Gatos · On-site

← All jobs
About Netflix

Watch Netflix movies & TV shows online or stream right to your smart TV, game console, PC, Mac, mobile, tablet and more. Backed by IVP and Redpoint.

About the role

The AI Observability team makes AI, ML, and Agentic systems transparent, reliable, and production-ready at scale. We build end-to-end observability for ML and GenAI workloads, capturing model inputs, features, predictions, outcomes, and behavior across online and batch systems. Our platform enables teams to monitor model performance, data quality, drift, latency, and failures, turning the ML system from a black box into an explainable, debuggable system.

What they're looking for

  • Experience in software, AI/ML, or platform engineering, with hands-on time in production observability, monitoring, or ML/LLM evaluation
  • Proven track record of designing standards, libraries, SDKs, or frameworks that other teams build on and adopt, reflecting a platform and enablement mindset rather than shipping features for a single use case
  • Strong coding skills in Python and at least one of Java, Go, or Scala, with experience building production services
  • Practical experience operating ML models in production (online serving and/or batch), including drift, model performance, and data quality
  • Hands-on experience with modern observability stacks (e.g., Prometheus/Grafana, Datadog, OpenTelemetry, ELK/OpenSearch, Jaeger/Tempo, or similar)
  • Solid understanding of distributed systems, microservices, and at least one major cloud platform (AWS, GCP, or Azure)
More about this role

At Netflix, our mission is to entertain the world. Together, we are writing the next episode - pushing the boundaries of storytelling, global fandom and making the unimaginable a reality. We are a dream team obsessed with the uncomfortable excitement of discovering what happens when you merge creativity, intuition and cutting-edge technology. Come be a part of what’s next.

AI and ML power innovation in all areas of the business, including helping members choose the right title for them through personalization, better understanding our audience and our content slate, creating high-quality subtitles, dubbings, images, trailers, and other assets, optimizing our payment processing, and much more. AI Platform (AIP) organization builds highly scalable, differentiated AI infrastructure to maximize the business impact of all AI/ML practitioners at Netflix, which is key to accelerating this innovation.

The AI Observability team makes AI, ML, and Agentic systems transparent, reliable, and production-ready at scale. We build end-to-end observability for ML and GenAI workloads, capturing model inputs, features, predictions, outcomes, and behavior across online and batch systems. Our platform...

Read the full posting on Netflix's site ↗

Build your edge while you search

Free tools for founders and investors, plus VC Unfiltered, our take on startups, venture and the people who build them.