Cut OpenAI, Anthropic, and Gemini API costs with accuracy held flat — or take a smaller cut and lift accuracy by several points instead. The bear-2 prompt compression API strips low-signal tokens from your inputs before they hit the LLM. Works with GPT,... Backed by Y Combinator.
About the role
As a Member of Technical Staff on our infrastructure team, you'll own the cloud systems that serve our compression API end-to-end. You'd get to build global low-latency, high-throughput GPU ML inference infra that sits in the critical path of customer traffic, from deployment and scaling to reliability and cost-efficiency. This is a very high ownership role where you'd need to learn a lot of new solutions and technologies, constantly improving and researching what's best.
More about this role
The Token Company (YC W26, HF0 S26) is a seed stage startup in San Francisco, CA, and has raised ~$12M from First Round Capital, NEA, YC and SV Angel along with investors such as founders of Dropbox, Slack, Supercell and Huggingface and key people from OpenAI, xAI and DoorDash.
The Token Company trains machine learning models to compress raw LLM inputs before they are passed to the expensive LLMs. Our models are able to recognize parts of inputs that are unimportant, reducing inference costs for scale-ups and enterprises that integrate LLMs into their products. The team is 5 people with a research and product focus.
As a Member of Technical Staff on our infrastructure team, you'll own the cloud systems that serve our compression API end-to-end. You'd get to build global low-latency, high-throughput GPU ML inference infra that sits in the critical path of customer traffic, from deployment and scaling to reliability and cost-efficiency. This is a very high ownership role where you'd need to learn a lot of new solutions and technologies, constantly improving and researching what's best. The ideal candidate fits our culture and is:
A quick learner: grasping the product and the systems...
Browse similar: AI jobs · AI startup jobs · Startup jobs · San Francisco Bay Area