Databricks Spark Data Engineer Opening

Take on a vital Senior Data Engineer role at Pacer Group, specializing in Databricks and Apache Spark. Spearhead the migration of legacy systems to a robust Data Lakehouse architecture.

This position involves hands-on work to modernize Cloudera technologies, including transforming existing platforms to Databricks Lakehouse. You'll focus on designing scalable architectures and crafting high-volume PySpark and Spark SQL ELT pipelines while optimizing for performance and cost efficiency.

Key Responsibilities: • Implement Databricks Lakehouse from legacy Cloudera systems • Create scalable architectures for Data Warehouses and Delta Lakes • Develop and optimize PySpark/Spark SQL ELT data pipelines • Enhance performance and cost-effectiveness of workloads • Work with complex Finance and Risk data models for reporting

Requirements: • Proven hands-on experience with Databricks and Apache Spark • Expertise in Cloudera ecosystems (CDH/CDP, Hive, HBase) • Knowledge of Finance and Risk data frameworks • Ability to design efficient data architectures • Strong skills in PySpark and Spark SQL

Drive innovation at Pacer Group by utilizing your Databricks and Apache Spark expertise in a pivotal engineering role.