# Software Engineer, Inference Systems at River AI

- Company: River AI
- What the company does: Develop frontier language models and agents. Training, reinforcement learning, and inference in one system, on River Cloud or your own GPU cluster. Backed by General Catalyst.
- Company website: https://river.ai/
- Type: Startups (AI role)
- Level: Mid level
- Location: Palo Alto, CA
- Work setup: On-site
- Pay: $200K to $420K base salary per year (USD)
- Posted: 2026-09-11
- Apply by: 2026-10-26
- Apply: https://job-boards.greenhouse.io/riverai/jobs/4402451009
- Page: https://www.1752.vc/careers/jobs/river-ai-software-engineer-inference-systems/

## About the role

We are looking for exceptional inference systems engineers to build the engines that serve large models through the River API. Your goal is to deliver fast, reliable inference while making efficient use of GPU compute and memory. You will take ownership of the serving runtime, from request scheduling and continuous batching to KV-cache management, distributed model execution, and checkpoint loading. Your work will support both customer-facing inference and the sampling workloads that power reinforcement learning.

## What they're looking for

- Bachelor’s degree in Computer Science, Computer Engineering, or equivalent practical experience
- Experience building inference engines or performance-sensitive distributed services
- Strong understanding of transformer inference, GPU memory, concurrency, and networking
- Proficiency in Python and C++ or Rust
- Strong debugging and profiling skills across models, runtimes, and services
- A collaborative mindset and strong ownership of engineering outcomes

Tags: River API