The Senior Data Platform Engineer builds the platform-services layer that turns the enterprise data platform into a product — API Gateway baseline, knowledge graph, event stream, semantic layer, data-product catalogue, vector search, anti-corruption-layer patterns, and the Python / TypeScript SDK framework that internal product teams consume. This is a senior software-engineering role focused on shipping platform capability with API contracts, SDKs, and governance baked in.
Key Responsibilities
- Design, build, and operate the platform API Gateway baseline — authentication, rate limiting, observability, versioning, contract tests, deprecation policy.
- Build and operate the Knowledge Graph platform — schema design, ingestion, query API, scaling, observability — exposed as a platform service to internal consumers.
- Build and operate the Event Stream platform — domain-event topology, partitioning, schema registry, retention; integrate with downstream catalogue and KG.
- Build and operate the Semantic Layer service — business-meaningful query API on top of the lakehouse, consumed by internal product teams and AI agents.
- Build and maintain the Data Product Catalogue — data-product registration, freshness SLOs, schema contracts, governance hooks at publish time.
- Build and operate the Vector Search Service infrastructure (pgvector, Milvus, Qdrant, or comparable) — index, scaling, observability — partnered with AI engineering on embedding model selection and retrieval relevance.
- Author and maintain the Anti-Corruption-Layer Patterns Library — reusable patterns for legacy system integration (Oracle, SAP, WMS) via events and ACL only; never direct DB writes.
- Build and maintain the Python and TypeScript SDK framework — consumed by product teams and AI engineering for all platform-service calls.
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or a related discipline.
- 8+ years software engineering with production ownership of platform services at multi-team scale.
- Strong Python (production services, async, observability) and TypeScript at senior level.
- API-first design discipline — OpenAPI 3.1, REST, gRPC; versioning, semver, deprecation policies, contract tests.
- Production experience with at least one of: knowledge graphs (Neo4j, Neptune, TigerGraph), event streaming (Kafka, Event Hubs), or vector databases (pgvector, Milvus, Qdrant).
- Distributed systems fundamentals — concurrency, message queues, idempotency, observability, performance.
- SDK / client library design for multi-language consumption; semver, breaking-change policy, documentation discipline.
- Cloud + Databricks (or equivalent lakehouse) production experience; Azure preferred.
Preferred Qualifications
- Production knowledge-graph experience at scale (multi-million entities); semantic layer / data products in production.
- Multi-language SDK design across Python and TypeScript / Java; OSS library maintenance.
- Retail / commerce data integration — Oracle (RMS, RPM, PMS, SIM), SAP, WMS; strangler-fig patterns.
- Vendor certifications such as Databricks Data Engineer Professional, Azure Solutions Architect, or comparable.