# Tech Lead Manager, Inference at Luma AI

- Company: Luma AI
- Role group: Research & AI
- Location: Redwood City
- Type: FullTime
- Remote: yes
- Posted: 2026-07-24
- Status: active
- Apply: https://jobs.ashbyhq.com/lumaai/b34cbe6c-aef5-4e9b-a6f7-987771e9140a?ref=companies-hiring.click

## Description

You'll lead the team that owns Luma's entire inference serving stack — routing, scheduling, and fleet-wide orchestration across thousands of GPUs, multiple clouds, and hardware vendors — where all of Luma's compute meets all of its users. This is a hands-on tech-lead-manager role.

It's leadership by shipping: at least half your time stays hands-on in the serving stack, alongside hiring, growing the team, and setting technical direction. It fits someone who's operated inference fleets at the thousands-of-GPUs scale and genuinely wants to keep building, not move into pure management. If you want a hands-off management seat, this isn't it.

What You'll Own

 - Spend at least half your time hands-on: architect and build core platform components, own the hardest design decisions, and debug the toughest incidents yourself.

 - Lead, grow, and develop the inference engineering team — hiring, coaching, on-call, incident response, capacity planning, and postmortems.

 - Set the technical roadmap for serving: engines, routing, scheduling, autoscaling, caching, observability, and deployment.

 - Own the platform's SLOs and economics: latency, availability, GPU utilization, and cost per generation.

 - Partner with research to ship new architectures to production on day zero and integrate serving into online RL and evaluation loops.

 - Build scheduling and queueing that leverages expensive GPU resources against live traffic, cluster availability, and user priority.

First 90 Days

One way the first 90 could unfold.

 - Days 1–30 — Immerse & Diagnose: Learn the serving stack, the team, and where reliability, latency, or cost hurt most.

 - Days 30–60 — Ship & Validate: Personally ship a meaningful platform improvement while setting the team's technical bar.

 - Days 60–90 — Scale & Systemize: Set the roadmap, grow the team, and harden SLOs and economics across the fleet.

What You Bring

 - 8+ years in large-scale distributed systems or ML infrastructure, with several years building and operating model-serving or inference platforms in production.

 - Experience running inference platforms at the thousands-of-GPUs scale across multiple clusters or clouds, and knowing what breaks there.

 - Technical leadership experience through rapid growth, with a genuine desire to stay at least half hands-on.

 - Deep expertise in LLM and foundation-model serving engines (vLLM, SGLang, TensorRT-LLM), ideally having modified engine internals.

 - Strong command of continuous batching, KV-cache management, quantization, speculative decoding, and parallelism strategies (TP/EP/pipeline).

 - Strong Python and PyTorch, Kubernetes at scale, and experience with queues, scheduling, traffic control, and fleet management.

Nice to Have

 - Experience serving diffusion, video, or other multimodal generative models, and with FFmpeg/multimedia processing.

 - Modern networking stacks — RDMA (RoCE, InfiniBand), NVLink — and multi-node serving topologies.

 - Experience across heterogeneous accelerators (NVIDIA, AMD, TPU, Trainium) and the porting and validation that comes with them.

 - Contributions to open-source serving infrastructure (vLLM, SGLang, Ray, Kubernetes ecosystem).

 - Systems-language depth (Rust, C++, CUDA/HIP) for kernel- and runtime-level optimization.

About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer.
---
Independent index; not affiliated with the companies listed. Applications happen on each company's own site. Contact: mail@companies-hiring.click