Senior AI Engineer - LLM Adoption Department (LLMAD)
rakuten · Tokyo, Japan ·
- Seniority
- Senior
- Category
- AI engineering
- Experience
- 6+ years
- Company size
- 1000+
rakuten · Tokyo, Japan ·
Stackblitz · Remote
Bank of America · Charlotte
ghr · Charlotte
Culture Amp · Melbourne
Senior AI Engineer embeds in Rakuten business units to design, build, and ship production LLM agents that replace third-party APIs using in-house models and tools.
Job Description:
Business Overview
Rakuten Group, Inc. is a global leader in internet services that empower individuals, communities, businesses, and society. Founded in Tokyo in 1997 as an online marketplace, Rakuten has expanded to offer services in e-commerce, fintech, digital content, and communications to approximately 1.7 billion members around the world. The Rakuten Group has nearly 32,000 employees and operations in 30 countries and regions. For more information visit
Department Overview
The AI & Data Division (AIDD) creates powerful, customer-focused search, recommendation, data science, advertising, marketing, price, and inventory optimization solutions to a variety of businesses in the Rakuten group. Our goal is to drive innovation by developing new products and capabilities that deliver significant impact over longer timeframes using AI.
The LLMAD team drives adoption of Rakuten AI and open-source LLMs hosted within our own environment. We're hiring AI Engineers who can be embedded directly inside business units to drive real adoption of in-house large language models in place of third-party alternatives.
Position:
Position Details
As a Senior AI Engineer you will own a business unit engagement end to end. You sit inside their engineering team and build with them: designing the agent, selecting and evaluating the model, writing the prompts and tool-use logic, and getting it live in production on our own infrastructure.
You are expected to operate independently - taking a vague business ask, turning it into a concrete and evaluable LLM task, and being accountable for it shipping. What you build also becomes reusable: the cookbooks, recipes, and reference implementations the rest of the team and other business units start from.
<Key Responsibilities>
・Own a business unit engagement end to end - embed in their engineering team, design the agent (task decomposition, tool and function calling, multi-step orchestration, state, retrieval, guardrails), and write production code in their codebase.
・Take agents from prototype to production and own what that requires: latency, cost, failure modes, and monitoring - including the judgment to say early when an agent is the wrong answer and a prompt, a fine-tune, or plain software is not.
・Select the right in-house or open-source model for each task and prove it with task-level evaluation rather than benchmark scores, building the eval sets and harnesses together with the business unit.
・Run migrations from third-party APIs to our in-house models - parity testing, prompt porting, staged cutover - and feed the gaps you find back to our model and platform teams.
・Own prompt and context engineering as a discipline: structured output, tool schemas, retrieval strategy, and caching, versioned and measured against evals; debug quality problems down to whether the fault is the prompt, the retrieval, the model, or the data.
・Turn the work into cookbooks, recipes, and reference implementations other business units can lift directly, and raise the bar around you through code review, mentorship, and hands-on enablement for business unit engineers.
<Scope of the Role>
This role sits close to the product rather than close to the model:
・You will not be training or fine-tuning foundation models - that sits with our model research and training teams.
・You will not be building the LLM platform itself - that sits with our platform engineering teams.
・You will be embedded in a business unit, writing agent and integration code inside their product, and accountable for it shipping on our models.
Mandatory Qualifications:
・Over 6 years of professional software engineering experience, with recent hands-on focus on building LLM applications. You are shipping this work today.
・Proven agentic engineering experience: you have built, shipped, evaluated, and debugged agents in production - tool and function calling, multi-step orchestration, and the failure modes that come with them.
・Prompt and context engineering as a discipline: structured outputs, systematic iteration against evals, and versioning.
・Practical model knowledge: the strengths and weaknesses of open-weight and commercial models, when to fine-tune versus prompt versus retrieve, and the cost and latency trade-offs of each.
・Solid production fundamentals: APIs, backend services, testing, observability, and experience with self-hosted or cloud inference (e.g., vLLM).
・Able to work embedded in someone else's engineering team, communicate directly with business stakeholders, and operate with the ambiguity of a function that is still being defined.
・Writes things down - docs, examples, and recipes other engineers can actually use.
Desired Qualifications:
・Japanese language ability - a significant plus. Our business unit teams work in Japanese day to day, and being able to work in the language directly makes the embedded model far more effective.
・Experience migrating production applications from third-party LLM APIs to self-hosted or open-weight models.
・Retrieval-augmented generation at production scale.
・Exposure to fine-tuning or post-training, enough to know when to reach for it.
・Experience delivering with distributed teams in Japan or Asia.
#engineer #applicationsengineer #aianddatadiv
replicant · Remote