All jobs Remote Browse Skills

Search jobs collected from company career pages and public sources, in one place.

Built by Darshit Gajjar. Follow me:

Explore

All jobs Browse all categories Remote jobs Junior roles Senior roles

Popular

AI engineering Frontend Backend Data science

Legal

Disclaimer & Terms Job safety tips Report a job

Jobyssey is a job search engine. We are not the employer or a recruiter and do not verify listings. Jobs link to third-party sites and you apply there at your own risk. Never pay to apply for a job. Company names and trademarks belong to their owners. See the full disclaimer.

© 2026 Jobyssey

Home Jobs Remote Explore

Member of Technical Staff, Inference & Serving at Inception – San Mateo | Jobyssey

Home
Jobs
Software engineering jobs
Member of Technical Staff, Inference & Serving

Member of Technical Staff, Inference & Serving

Inception · San Mateo, USA · 7 months ago

Apply on company site

Before you apply

Jobyssey is a search engine. We don't post, own or verify this job and are not the employer or a recruiter. You'll apply directly on jobs.gem.com, under that site's terms.

Never pay to apply, for training or equipment, and never share bank details, ID documents or passwords before you've confirmed the employer is real.

Job safety tips Report this job Disclaimer

Category: Software engineering

airflow aws azure ci-cd cloud containerization cuda distributed-systems docker gcp kubeflow kubernetes llm machine-learning

Explore more jobs

Software engineering jobs Remote Software engineering jobs Jobs in United States Software engineering jobs in United States

workflow-orchestration

The Role

We're looking for engineers and scientists to design, optimize, and scale the systems that power our diffusion LLMs in production. Your work will make inference faster, more cost-effective, and more reliable.

Key Responsibilities

Build and optimize high-performance model serving systems for low-latency inference of diffusion LLMs.
Extend orchestration frameworks (Kubernetes, Ray, SLURM) for distributed inference, evaluation, and large-batch serving.
Implement and manage load balancing, autoscaling, and traffic routing for model endpoints.
Build systems for model versioning, canary deployments, and zero-downtime rollouts.
Develop monitoring, alerting, and observability tooling to ensure SLA compliance and rapid incident response.
Collaborate with ML researchers to translate model advances (new architectures, quantization techniques, batching strategies) into production-ready serving improvements.

Qualifications

BS/MS/PhD in Computer Science, Engineering, or a related field (or equivalent experience).
Knowledge of ML serving frameworks (SGLang, vLLM, Triton Inference Server, TensorRT-LLM).
Understanding of ML frameworks (PyTorch, TensorFlow) from a systems perspective.
Familiarity with high-performance computing and GPU programming (CUDA).
Experience with containerization (Docker), orchestration (Kubernetes), and CI/CD pipelines.
Background in performance optimization and profiling of ML systems.

Preferred Skills

Experience building and maintaining large-scale language models with tens of billions of parameters or more.
Experience with distributed systems and cloud computing platforms (AWS/GCP/Azure).
Experience with ML workflow orchestration tools (Kubeflow, Airflow).
Experience with model optimization techniques (quantization, distillation, speculative decoding, continuous batching).
Knowledge of ML-specific infrastructure challenges (checkpointing, resource scheduling, etc.).