# Member of Technical Staff - Web Crawl Engineer at Reflection AI

- Company: Reflection AI
- What the company does: Make intelligence open and accessible to all. Backed by Battery, Lightspeed and Sequoia.
- Company website: https://reflection.ai/
- Type: Startups (AI role)
- Level: Senior
- Location: San Francisco, CA
- Work setup: On-site
- Posted: 2026-06-19
- Apply by: 2026-10-08
- Apply: https://jobs.ashbyhq.com/reflectionai/ee258570-a3f9-4276-9bf4-35b2b70dbc61
- Page: https://www.1752.vc/careers/jobs/reflection-ai-member-of-technical-staff-web-crawl-engineer/

## About the role

The web is one of the most important sources of information for frontier AI systems. The quality, coverage, freshness, and diversity of web data directly influence model capabilities.

## What they're looking for

- Experience building large-scale web crawling, search indexing, content acquisition, or internet-scale data collection systems
- Strong understanding of crawling architectures, URL frontier management, scheduling, and distributed crawl coordination
- Experience with large-scale distributed systems using technologies such as Ray, Spark, Beam, Flink, or similar frameworks
- Familiarity with content extraction, HTML parsing, browser automation, rendering systems, and modern web technologies
- Experience operating systems that process petabyte-scale datasets
- Strong systems engineering skills, including reliability, observability, performance optimization, and debugging

Tags: Research