Principal / Lead Data Engineer - Finance (all genders)
BESTSECRET ·
- Seniority
- Principal
- Category
- Data engineering
- Experience
- 5+ years
BESTSECRET ·
Sabert · Chicago, IL, USA
Capital Technology Group · Remote
EXL · United States
Senior individual-contributor role owning BESTSECRET's finance data platform: integrating SAP financial data (FI/CO, MM, PTP) into a data lake, building ETL/ELT pipelines and warehouse models with Airflow, Kubernetes, Starburst, dbt, Kafka, Iceberg and Microsoft Fabric, and ensuring line-level reconciliation and data quality for reliable financial reporting.
As a Principal / Lead Data Engineer with a focus on finance and SAP-related data, you will be responsible for the design, development, and evolution of our finance data platform. You will build reliable data architectures, ensure accurate reconciliation between source systems and the data lake, and support high-quality financial reporting. Working closely with engineering teams and Finance stakeholders, you will help develop a scalable, maintainable data platform with a strong focus on performance, reliability, and auditability. This is a senior individual contributor role with technical leadership responsibilities but no direct people management.
Key Responsibilities
Provide strategic and technical leadership for finance-related data domains, driving improvements in data quality, reconciliation processes, and platform efficiency.
Own the end-to-end integration of SAP financial data, including FI/CO, MM, and PTP processes, into the data lake, and design and maintain scalable ETL/ELT pipelines for batch and near-real-time financial data using technologies such as Airflow, Kubernetes, Starburst, dbt, and Kafka.
Design, extend, and optimize the enterprise data warehouse and broader data platform using technologies including Starburst, Iceberg, Microsoft Fabric, and Looker, while partnering with Finance and analytics stakeholders to deliver reliable, high-performance reporting and data models that support financial reporting and decision-making.
Establish reconciliation frameworks, data quality controls, and automated validation processes to ensure accurate, line-level reconciliation between SAP and lakehouse data and improve confidence in financial reporting.
Lead automation initiatives, including pipeline code generation and CI/CD enhancements using GitLab pipelines.
Your Profile
5+ years of hands-on experience designing and implementing scalable data solutions, ideally with exposure to finance or accounting data.
Degree in Computer Science, Mathematics, Engineering, Physics, or a related field.
Strong SQL expertise, ideally with dbt, a solid understanding of relational and columnar databases, and proficiency in Python.
Experience working with SAP financial data structures and extraction, ideally within FI/CO, MM, or comparable modules. You do not need to be an SAP specialist but should be comfortable working with SAP data and understanding how it supports financial reporting and reconciliation.
Experience with modern data technologies such as Starburst, Databricks, Snowflake, or similar.
Experience working with Linux environments and cloud platforms such as AWS, Azure, or similar.
Strong communication skills in English, with an interest in emerging technologies and continuous learning. Spanish is a plus.
Ability to translate Finance requirements into robust technical solutions.
#LI-MT1 #LI-HybridMalaga
EXL · Pune, Maharashtra, India