We are looking for a
Python Developer (Data Engineering) to design, build, and support scalable data pipelines and analytics platforms within a large enterprise banking environment. The role involves end-to-end data engineering across batch and near-real-time processing, strong collaboration with business and IT stakeholders, and adherence to high standards of quality, governance, and security.
Key Responsibilities
Develop robust data ingestion, transformation, and loading (ETL/ELT) processes across batch and near-real-time workflows
Implement distributed data processing using Apache Spark (PySpark / Scala) for large-scale transformations and analytics
Design and maintain logical and physical data models (dimensional/star schemas, data vault, wide tables) optimized for reporting and analytics
Write and optimize SQL and HiveQL queries; manage tables, partitions, and storage formats
Schedule, monitor, and support data pipelines using Control-M, ensuring SLA adherence and timely delivery
Perform performance tuning of Spark jobs, SQL/Hive queries, and storage strategies for scalability and cost efficiency
Implement data validation, reconciliation, and lineage using checks, unit tests, and metadata frameworks
Build operational dashboards and alerts, investigate failures, and drive root-cause analysis and remediation
Maintain comprehensive runbooks, architecture diagrams, data dictionaries, and coding standards
Apply data privacy, access control, and security best practices in line with enterprise and regulatory requirements
Drive continuous service and process improvements
Prepare unit test cases and work closely with Testing teams during SIT and UAT
Package and migrate code across environments (DEV → QA → PROD) with proper audit trails and governance