# Staff Software Engineer, AI Inference Gateway at Salad

- Company: Salad
- What the company does: Our mission is to democratize cloud computing by utilizing latent consumer resources to power a sustainable, affordable, environmentally friendly cloud for everyone. salad.com/about. Backed by Techstars.
- Company website: https://saladafrica.com
- Type: Startups (AI role)
- Level: Senior
- Location: Remote (United States)
- Work setup: Remote
- Pay: $180K to $220K base salary per year (USD)
- Posted: 2026-09-18
- Apply by: 2026-11-02
- Apply: https://ats.rippling.com/salad/jobs/699d9ff3-5bbb-4a6a-a482-c933196c4e91
- Page: https://www.1752.vc/careers/jobs/salad-staff-software-engineer-ai-inference-gateway/

## About the role

SaladCloud runs inference on thousands of geodistributed workstation and consumer GPUs — NVIDIA and AMD — that nobody else can use. Sitting in front of that fleet is a unique AI Gateway written in Rust, built on Pingora and WireGuard, that serves real-time and batch inference and embeddings to customers as a single, reliable API. You'll own it.

## What they're looking for

- Strong production Rust experience, ideally on high-throughput networked services
- Hands-on experience running LLM inference servers (vLLM, llama.cpp, TGI, TensorRT-LLM, or similar) — you know what a KV cache is and why it fills up
- Distributed systems fundamentals: load balancing, backpressure, failure handling, unreliable nodes
- Comfort operating what you build: metrics, tracing, on-call instincts
- Willing to be on-call for a system you helped make quiet — we treat after-hours pages as a design bug to fix, not a lifestyle
- Clear written and verbal communication, you can run a design without hand-holding and bring others along

Tags: SaladCloud