Back to search
About the Role
Join a small AI infrastructure team focused on improving the speed and cost efficiency of large language model inference. You will work across model serving, GPU kernels, and production infrastructure, with meaningful ownership from customer conversations through implementation.
What You'll Do
Deliver day-one support for new open-source models, tuning them for latency and throughput.
Optimize model-serving systems, including batching, KV cache, speculative decoding, and quantization.
Write and tune kernels using CUDA, HIP, and Triton across a range of accelerator platforms.
Design, deploy, and operate heterogeneous clusters spanning multiple hardware vendors.
Run production inference systems, improving reliability, observability, and cost per token at scale.
Work with customers to understand needs and help shape technical solutions.
What We're Looking For
Typically 1 to 6 years of experience in backend, infrastructure, systems, or related engineering work, including early-career candidates with strong internships.
Strong understanding of computer architecture, memory hierarchy, and operating system internals beyond application-level development.
Hands-on experience with low-level or systems engineering, alongside any full-stack or web development experience.
Familiarity with CUDA, HIP, Triton, Python, or GPU programming, and experience with inference optimization techniques such as quantization, batching, KV cache, or speculative decoding.
Experience deploying or operating infrastructure at scale, such as data centers, clusters, or large GPU fleets.
Strong computer science fundamentals and a track record of tackling complex systems problems.
Comfort using AI agents and coding tools to improve engineering workflows, and communicating with customers as part of technical work.
Compensation & Benefits
Base salary is $200,000 to $300,000 USD annually. Visa sponsorship is available.
Location
On-site in San Francisco, California, five days per week.
Learn more about this Employer on their Career Site
