Design and maintain AI-ready data architectures, including vector and graph databases, to power retrieval-augmented generation systems and enterprise analytics for investment and market intelligence.
Design and build data architectures optimised for AI agent consumption, including structured stores exposed via APIs, vector databases for semantic retrieval, graph databases for relationship reasoning, and hybrid retrieval systems combining keyword, semantic, and structured queries.
Own the data layer for RAG pipelines: document ingestion workflows, chunking strategies, embedding generation and refresh, metadata tagging, and vector index management across domains (e.g., company research, market intelligence, portfolio data, regulatory filings).
Establish ontology and schema standards to ensure AI-accessible data is consistent, well-documented, and interpretable without custom parsing logic.
Architect real-time and near-real-time data feeds (e.g., market data, news, portfolio events), defining and enforcing latency and freshness SLAs.
Enterprise data quality and governance
Define and implement data quality standards (completeness, consistency, freshness, anomaly detection) with automated quality gates to prevent degraded data entering AI systems.
Build end-to-end data lineage tracking across AI pipelines, enabling traceability from source to AI consumption for debugging and audit requirements.
Partner with AI Security & Governance and enterprise data teams to ensure compliance with data classification, access control, and cross-border handling requirements (including China-related workflows).
Design and operate data observability tooling covering pipeline health, data drift, schema changes, and SLA monitoring, giving product teams visibility into data reliability.
Run regular data quality reviews with AI product teams to identify gaps impacting performance and prioritise data engineering investments.
Shared, reusable data platform for AI
Develop reusable data assets and services supporting multiple AI products, including a shared investment knowledge graph, company/market data APIs, document intelligence pipelines, and portfolio analytics services.
Maintain a data catalogue documenting sources, schemas, freshness, quality metrics, access protocols, and limitations to enable informed data usage.
Contribute to enterprise data platform strategy with an AI-first perspective, ensuring architectures support AI consumption patterns beyond traditional BI/reporting needs.
Engage external data vendors to evaluate, onboard, and maintain high-quality data sources, including ongoing quality assessment and licensing management with procurement.