Founding Senior AI Infrastructure Engineer

Nice to have skills: Terraform, Helm, Ansible, Ray

An AI infrastructure company specializing in inference optimization for large-scale models. The platform enhances token throughput and reduces latency by optimizing hardware utilization and decoding processes within a customer's cloud environment. The technology focuses on improving the efficiency of model serving without requiring changes to existing infrastructure or code, ensuring data privacy while reducing operational costs for organizations deploying open-source models.

About the Role

As a founding Senior AI Infrastructure Engineer, you will report to the Chief Technology Officer to design and operate large-scale infrastructure for AI workloads on public cloud and on-premises environments. You will be an individual contributor with significant influence, combining software engineering with deep systems expertise to build secure and reliable platforms. Your work will focus on enabling efficient model serving at scale, ensuring the infrastructure can support a massive number of concurrent users while maintaining high service quality and performance.

What You'll Do

• Design and manage multi-cloud and on-premises infrastructure for internal and external workloads.
• Orchestrate large-scale deployments using infrastructure-as-code and container orchestration to support high user concurrency.
• Benchmark AI inference workloads to identify performance bottlenecks and optimize hardware utilization.
• Develop and maintain comprehensive monitoring systems to ensure high service quality and reliability.
• Collaborate with customers to deploy and integrate optimization solutions within their existing infrastructure.
• Build and maintain an internal laboratory for infrastructure testing and experimentation.

What You Bring

• 5+ years of experience in AI infrastructure or systems engineering.
• Proficiency in Python and experience building and operating scalable APIs.
• Extensive experience with Kubernetes, Helm, Ansible, and Terraform.
• Proven track record of shipping large language model (LLM) serving systems in production environments.
• Experience with distributed computing frameworks such as Ray, RayClusters, or KubeRay.
• Demonstrated experience with agentic workflows.
• Bachelor’s degree in Computer Science or a related technical field.
• Founding-engineer equity and direct ownership.
• Remote-friendly work environment. Visit the office 2-3x a quarter in Montreal.
• Access to extensive compute resources and specialized hardware for experimentation.

About your mission

You’ll join a cutting edge team that is building next-generation infrastructure solutions to help organizations scale AI more efficiently. As AI workloads continue to grow, the team is focused on optimizing performance, reducing compute costs, and improving energy efficiency across the AI stack. Founded by experienced AI researchers and entrepreneurs, the company works across hardware, software, and model-serving technologies to deliver faster, more scalable AI systems.

This is an exciting chance to join a highly technical team working at the forefront of AI infrastructure innovation. It's an opportunity of a lifetime!

About the client

An early-stage AI infrastructure startup specializing in inference optimization and hardware efficiency. The company develops technology that monitors power and latency at the kernel level to maximize compute per watt. This enables high-throughput model serving within existing power constraints without necessitating software rewrites. By optimizing across the hardware and software stack, the platform significantly reduces the energy intensity and cost of scaling sophisticated AI models.