Technical Consultant - Python & Kafka
Fujitsu ·
- Category
- Solutions engineering
- Experience
- 5+ years
- Company size
- 1000+
Fujitsu ·
Cognite - AI for Industry · India (Bengaluru)
Kontakt.io · Kraków, Poland
flutterbe · Porto, Portugal
Bitovi · Chicago, IL
Hybrid senior engineer role in Pune working UK-hours on an event-driven digital repository platform: hands-on Python services, Kafka reliability (offsets, consumer groups, DLQs, delivery semantics), PostgreSQL/MongoDB, structured logging and observability (OpenTelemetry, OpenLineage/Marquez), with deployments via Kubernetes, Kustomize, and ArgoCD.
Job Role: Technical Consultant - Python & Kafka
Shift: UK Shift (2.00 PM to 10.00 PM IST)
Work Mode: Hybrid
Relevant Exp: 5+ Years
Job Description:
Position Summary
Hands-on Senior Engineer responsible for enhancing an existing event-driven digital repository platform, with focus on Kafka reliability, Python services, structured logging, event lineage, reconciliation, and completion tracking. This is an implementation-focused role working within an established architecture.
Key Responsibilities
Develop and maintain Python services using Kafka, PostgreSQL, MongoDB, object storage, and REST APIs.
Implement structured JSON logging, correlation, error context, and actionable diagnostics.
Improve Kafka reliability including offsets, consumer groups, rebalancing, retries, DLQs, and message-loss investigation.
Implement event tracking, lineage, reconciliation, and workflow completion verification.
Build observability using OpenTelemetry and contribute to OpenLineage/Marquez implementation.
Apply event-driven patterns including idempotency, delivery semantics, and failure recovery.
Write automated tests and support deployments in Kubernetes, Kustomize, and ArgoCD.
Troubleshoot distributed processing issues across services, messaging, databases, and external systems.
Must Have
Python – Advanced production development; concurrency/threading; testing and maintainable code.
Apache Kafka – Producers/consumers, offsets, consumer groups, rebalancing, retries, DLQ, delivery semantics.
PostgreSQL – Strong SQL and JSON/JSONB capabilities; operational/event data.
Structured Logging & Observability – JSON logging standards, correlation, severity, diagnostics, distributed-system observability.
Good to Have
OpenTelemetry
OpenLineage / Marquez
Kubernetes, Kustomize & ArgoCD
Grafana / Grafana Loki
Event-driven architecture – idempotency, at-least/exactly-once concepts, transaction boundaries, failure recovery.
Nice to Have
MongoDB
AWS S3 / S3-compatible storage
REST APIs and distributed data-processing platforms
Experience with digital repository, media, or data-processing workflows.
mirantis · Remote, REMOTE, us