# Distributed LLM Inference Engineer at Anyscale

- Company: Anyscale
- What the company does: Powered by Ray, Anyscale helps AI builders run data-intensive workloads to build and deploy Foundation Models and AI at scale on any cloud. Backed by NEA, a16z and Amplify.
- Company website: https://www.anyscale.com/
- Type: Startups (AI role)
- Level: Mid level
- Location: San Francisco
- Work setup: Remote
- Pay: $170K to $245K base salary per year (USD)
- Posted: 2026-05-27
- Apply by: 2026-10-08
- Apply: https://jobs.ashbyhq.com/anyscale/1cf38233-8aa0-47f8-9d85-65ce27bc3047
- Page: https://www.1752.vc/careers/jobs/anyscale-distributed-llm-inference-engineer/

## About the role

As a Distributed LLM Inference Engineer, you will help systems and optimizations that push the boundaries of performance for inference at large scale. This is an incredibly critical role to Anyscale as it allows us to achieve a market leading position for AI infrastructure.

## What they're looking for

- Familiarity with running ML inference at large scale with high throughput and low latency
- Familiarity with deep learning and deep learning frameworks (e.g. PyTorch)
- Solid understanding of distributed systems, ML inference challenges

Tags: Engineering