Company:
Marsh
Description:
Marsh Management Consulting is looking for a hands-on Senior Software Engineer with strong software engineering fundamentals and deep practical experience building production Generative AI applications. The role will help build and own the next generation of AI-powered experiences across Marsh Management Consulting digital platforms.
The successful candidate will help establish reusable internal AI capability spanning retrieval, orchestration, evaluation, guardrails, content ingestion, APIs, observability, and integration with the website experience. The role will work closely with product, UX, content, AEM/web engineering, architecture, analytics, security, and external delivery partners, while building the knowledge required for Marsh Management Consulting to own and evolve the platform over time.
We will count on you to:
AI engineering and architecture
- Design, build, and operate production-grade LLM applications using retrieval-augmented generation (RAG), semantic search, prompt orchestration, and agentic patterns where appropriate.
- Build conversational experiences incorporating intent detection, session context, multi-turn interactions, structured outputs, tool/function calling, and streaming responses.
- Develop reusable AI services and APIs rather than tightly coupling AI logic to individual website features.
- Own technical decisions around model selection, retrieval strategy, orchestration, performance, and AI application architecture.
Search, content and RAG
- Design retrieval strategies using embeddings, vector search, keyword/hybrid retrieval, metadata filtering, reranking, and contextual retrieval.
- Build and maintain content ingestion pipelines that crawl, process, chunk, enrich, embed, and index Oliver Wyman website content.
- Work with content and AEM teams to use taxonomy, tagging, metadata, and content lifecycle signals to improve retrieval quality and control AI inclusion and exclusion.
- Implement source attribution and citation mechanisms so generated answers can be traced to authoritative Oliver Wyman content.
- Continuously improve relevance and retrieval performance using quantitative evaluation and user feedback.
AI quality, evaluation and safety
- Build systematic evaluation frameworks for retrieval accuracy, groundedness, answer quality, brand and tone alignment, and safety.
- Create automated evaluation datasets and regression testing so model, prompt, and retrieval changes can be assessed before deployment.
- Implement safeguards against hallucination, prompt injection, inappropriate content, PII exposure, and ungrounded responses.
- Establish feedback loops and monitoring that enable continuous improvement of AI quality.
AWS and production engineering
- Design and operate cloud-native AI services on AWS, ideally using services such as Amazon Bedrock, OpenSearch, Lambda, API Gateway, S3, ECS/Fargate, DynamoDB, and Redis.
- Build secure, observable, and scalable services with appropriate logging, tracing, monitoring, and auditability.
- Work with infrastructure teams on CI/CD and Infrastructure-as-Code using technologies such as Terraform and GitHub Actions.
- Optimize AI services for reliability, latency, scalability, and cost.
Digital experience integration
- Partner closely with AEM and front-end engineers to integrate AI capabilities into the Oliver Wyman website and its transition to AEM Cloud.
- Design APIs and event/context models that allow the AI Companion to understand the current page, content, and user journey.
- Enable conversational search, assisted navigation, contextual prompt suggestions, relevant content and tool recommendations, contextual calls-to-action, and TL;DR summaries.
- Support the evolution from an AI assistant layered over the website toward AI-enhanced components and more personalized digital experiences.
What you need to have:
- Approximately 6+ years of professional software engineering experience, with meaningful recent experience delivering Generative AI applications into production.
- Strong Python engineering skills for AI and backend development.
- Demonstrable production experience building RAG and semantic-search systems.
- Strong understanding of embeddings, chunking, vector databases, hybrid search, metadata filtering, retrieval optimization, and ideally reranking.
- Hands-on experience integrating LLMs through platforms or APIs such as Amazon Bedrock, Azure OpenAI, OpenAI, or equivalent.
- Experience building conversational or agentic applications involving multi-turn context, orchestration, structured outputs, tool calling, and/or streaming.
- Strong API and backend engineering capability.
- Practical cloud experience, ideally AWS.
- Experience with automated testing, CI/CD, production observability, and secure software development.
- Demonstrable experience evaluating LLM/RAG quality rather than relying solely on manual prompt testing.
- Understanding of GenAI security and safety, including grounding, prompt injection, PII/data protection, and guardrails.
- Ability to work across engineering, product, UX, content, architecture, analytics, and security disciplines.
- Strong communication skills with the ability to explain complex AI engineering decisions to technical and non-technical stakeholders.
What makes you stand out:
- Amazon Bedrock and AWS AI services.
- OpenSearch or comparable enterprise/vector search platforms.
- MLflow or equivalent GenAI observability and evaluation tooling.
- Terraform and GitHub Actions.
- Experience working with large web/content estates or enterprise CMS platforms.
- AEM / AEM as a Cloud Service familiarity.
- Content taxonomy and metadata-driven retrieval.
- React/TypeScript and experience implementing user-facing AI experiences.
- Personalization or recommendation systems.
- Adobe Analytics or comparable digital analytics platforms.
Why join our team:
- Work in an environment that empowers individuals to reach their full potential, fostering personal and professional growth
- We help you be your best through professional development opportunities, interesting work, and supportive leaders;
- We foster a vibrant and inclusive culture where you can work with talented colleagues to create new solutions and have an impact for colleagues, clients, and communities;
- Our scale enables us to provide a range of career opportunities, as well as benefits and rewards to enhance your well-being;
- A yearly budget and the opportunity to build your flexible benefits package (up to 20% of your annual salary);
- 30+ days off (including legal days, birthday, public holiday replacements, and benefits options);
- Performance bonus scheme;
- Matching charity contributions, charity days off, and the Pay it Forward charity challenge;
- Core benefits: Pension, Life and Medical Insurance, Meal Vouchers, Travel Insurance
Marsh (NYSE: MRSH) is a global leader in risk, reinsurance and capital, people and investments, and management consulting, advising clients in 130 countries. With annual revenue of over $27 billion and more than 95,000 colleagues, Marsh helps build the confidence to thrive through the power of perspective. For more information, visit marsh.com, or follow us on LinkedIn and X.
Marsh is committed to creating a diverse, inclusive and flexible work environment. We aim to attract and retain the best people and embrace diversity of age, background, disability, ethnic origin, family duties, gender orientation or expression, marital status, nationality, parental status, personal or social status, political affiliation, race, religion and beliefs, sex/gender, sexual orientation or expression, skin color, or any other characteristic protected by applicable law.
Marsh is committed to hybrid work, which includes the flexibility of working remotely and the collaboration, connections and professional development benefits of working together in the office. All Marsh colleagues are expected to be in their local office or working onsite with clients at least three days per week. Office-based teams will identify at least one “anchor day” per week on which their full team will be together in person.