Director: Lead Data Engineer

Section / Function:Insert Section/Function

Travel Required:No

SMCR Status:N/A

Effective from:

Dual Hat Department:Non-employing entity Dual Hat Department

Dual Hat Reports to:Non-employing entity Manager’s Job Rank/Corporate title and functionaltitle

Dual Hat SMCR:Client Dealing Function, Material Risk Takers, Manager of Certified Employees, Significant Management Function, CASS Oversight function, Algorithmic trading or N/A

Profile Summary

We are seeking a highly skilled and experienced Lead Data Engineer to join our talented team. The ideal candidate will have a strong background in data engineering, with experience in AI and a proven track record of designing and implementing robust data pipelines.

Responsibilities

• Lead the design, development, and maintenance of scalable and efficient data pipelines to support our data-driven applications


• Collaborate with cross-functional teams, including data scientists, software engineers, and product managers, to understand data requirements and deliver high-quality solutions


• Architect and implement data ingestion, transformation, and storage processes using tools such as Apache Spark, Azure Data Factory, and other similar technologies


• Optimize data pipeline performance, ensuring data accuracy, reliability, and timely delivery


• Mentor and guide junior data engineers, fostering a culture of continuous learning and improvement


• Stay up-to-date with the latest trends and best practices in data engineering and AI, and apply them to our projects


• Participate in the evaluation and selection of new technologies and tools to enhance our data engineering capabilities


• Contribute to the development of AI-powered features and products, leveraging your knowledge of machine learning and data science concepts

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field


• 7+ years of experience in data engineering, with at least 3 years in a leadership role


• Strong proficiency in Apache Spark, Azure Data Factory, and other data pipeline tools


• Experience with cloud platforms such as Azure, AWS, or GCP


• Solid understanding of data modeling, data warehousing, and ETL/ELT processes


• Familiarity with AI and machine learning concepts, with hands\u2011on experience in implementing AI-driven solutions


• Excellent problem-solving skills and the ability to design and optimize data architectures


• Strong programming skills in languages such as Python, Scala, or Java


• Experience with data quality, data governance, and data security best practices


• Excellent communication and collaboration skills, with the ability to work effectively in a team environment


• Ability to thrive in a fast-paced, dynamic environment and manage multiple priorities

Preferred Qualifications

• Advanced degree in Computer Science, Data Science, or a related field


• Certifications in relevant technologies, such as Azure Data Engineer or Databricks Certified Developer


• Experience with real-time data processing and streaming technologies like Apache Kafka or Azure Event Hubs


• Knowledge of data visualization tools, such as Power BI or Tableau


• Contributions to open-source projects or active participation in the data engineering community