Applications: candidates must already be in Japan. Visa sponsorship: available. Japanese: Business level. English: Fluent. Japan Dev tags: Engineering, Eng - Other, Python, C++, Typescript, Hadoop, Openstack, Kubernetes, AWS, Spark, Docker, Github, Slack.
MISSION
As the founding MLOps engineer, design and build Shizuku’s ML infrastructure from the ground up. Establish the complete pipeline — from data ingestion through training environments to model serving — creating an internal platform that empowers ML engineers to iterate on models at maximum velocity.
Replace individual, siloed development environments with a unified team-scale ML development platform, maximizing the speed of Shizuku’s evolution.
ABOUT SHIZUKU
Shizuku is a Japan-born AI companion actively engaging audiences on YouTube and X (formerly Twitter). Already running live streams and cultivating a growing community, Shizuku is now entering its next phase of rapid scale.
As the first Japanese startup to receive investment from a16z, we closed our seed round and are on a mission to bring Japanese entertainment × AI to the global stage.
TEAM STRUCTURE
You will work closely with founder Aki (ML engineer and researcher, ex-Meta, ex-Luma AI) and Engineering Director Ohno to drive the design and construction of our ML infrastructure. As the first MLOps engineer, you’ll have significant autonomy — from technology selection to operational design.
Post-foundation, career paths include both a management track leading a growing team and an IC track deepening technical expertise, tailored to your aspirations.
CURRENT STATE & WHAT YOU’LL BUILD
- Infrastructure Status: Modern application infrastructure is in place, but ML training and MLOps tooling are not yet established. AWS adoption is planned
- What You’ll Build: An internal platform for ML engineers developing Shizuku’s AI models. The goal: eliminate siloed, ad-hoc local workflows and code ownership by individuals, replacing them with a team-oriented ML development foundation
KEY RESPONSIBILITIES
- Design, build, and operate the end-to-end ML training pipeline: data collection/preprocessing → training → evaluation → deployment
- Design and build GPU training infrastructure on AWS (A100, L4, etc.) with cost optimization
- Build an internal ML platform for engineers: experiment tracking, model versioning, and reproducibility guarantees
- Design and build model serving infrastructure: inference APIs, auto-scaling, and latency management
- Establish training data management and quality assurance pipelines
- Design and implement CI/CD for ML: automated training, model testing/evaluation, and staged rollouts
- Drive production integration of models in collaboration with ML Engineer and SWE teams
- Build monitoring and visibility infrastructure for long-term compute cost and GPU utilization tracking
REQUIREMENTS
- 3+ years of experience designing, building, and operating cloud infrastructure on AWS, GCP, or equivalent platforms
- Experience building ML/DL pipelines and infrastructure
- Hands-on experience designing and operating production environments using container technologies (Docker/Kubernetes)
- Experience managing infrastructure as code (Terraform, Pulumi, etc.)
- Strong Python skills for building tools and pipelines
- Ability to work on-site at our Tokyo office (primarily in-office with flexible remote arrangements)
NICE TO HAVE
- Experience building, operating, and cost-optimizing GPU clusters (A100, H100, L4, etc.)
- Experience with ML platforms: SageMaker, Vertex AI, Ray, Kubeflow, etc.
- Experience deploying and operating experiment tracking infrastructure: MLflow, Weights & Biases, DVC, etc.
- Experience building model serving infrastructure: Triton Inference Server, TorchServe, vLLM, SGLANG, etc.
- Experience designing and building internal ML development platforms
- Domain-specific knowledge of ML workloads in speech, NLP, or vision
- Experience as a founding infrastructure/MLOps engineer at a startup
- Technical communication skills in English (currently Japanese-first internally; transitioning to a global environment in the mid-term)
WHO YOU ARE
- Founding Engineer Mentality — You don’t wait for established systems to improve — you define the design philosophy and build the foundation from zero. You’re energized by creating the system itself, not just refining one
- ML-Literate Infrastructure Engineer — You understand the unique characteristics of ML training and inference workloads, and you translate that understanding into optimally designed infrastructure
- Purpose-Driven Ownership — You reverse-engineer from “maximizing ML team velocity,” set your own priorities, and drive execution autonomously
- Comfort with Ambiguity — You design for a world where model count, training frequency, and data volume are still being defined — starting small and scaling architecturally as the picture clarifies
- Resilience & Respect — You engage as an equal partner with ML Engineers and SWEs, elevating the entire team’s productivity through collaboration