Senior Data Engineer with Databricks

Key responsibilities

• Design, develop, and maintain scalable and reliable data pipelines for batch and/or real-time data processing.

• Develop and optimize data engineering solutions using Databricks and related cloud data technologies.

• Build and maintain data pipelines, ETL/ELT processes, and data transformation workflows.

• Work with large and complex datasets to ensure data quality, consistency, availability, and performance.

• Develop and optimize SQL and Python code for data processing and transformation.

• Implement data engineering solutions using Apache Spark / PySpark.

• Collaborate with Data Architects, Data Scientists, Analysts, Application Developers, and business stakeholders to understand data requirements.

• Troubleshoot and resolve data pipeline, integration, and performance issues.

• Apply best practices for data governance, security, monitoring, and data quality.

• Participate in technical design, code reviews, testing, deployment, and production support.

About you

• 8–10 years of experience in Data Engineering, Data Development, or a related field.

• Strong hands-on experience with Databricks in an enterprise data environment.

• Strong experience with Apache Spark / PySpark.

• Advanced proficiency in SQL and solid experience with Python.

• Strong understanding of ETL/ELT processes, data warehousing, and data pipeline development.

• Experience working with large-scale datasets and distributed data processing.

• Experience with cloud-based data platforms and services.

• Strong understanding of data quality, data modeling, and performance optimization.

• Experience with source control and software development best practices.

• Strong analytical, problem-solving, and troubleshooting skills.