Senior Data Engineer

The Opportunity

The Senior Data Engineer will play a critical role in our expanding Data Product Team. They will work hands-on with our entire company and contact profile universe with the main objective of improving the coverage, accuracy, and infrastructure scalability of our product data. This person will be tasked with solving significant data problems that are positively impacting millions of business professionals in the ever evolving lead intelligence space. The Senior Data Engineer will have the opportunity to work with cutting edge technology in big-data, ETL orchestration, data analytics, machine learning, and AI.

AWS-Based ETL Pipeline - Deep Data Engineering - Infrastructure Performance - Data Analytics

This is a hybrid data engineering and analytics role, not a pure pipeline-building position. Much of the work involves investigating data-quality issues, analyzing why current scoring rules produce bad outcomes, and deciding how they should change — that requires real analytical and statistical reasoning on top of the engineering work, not just building infrastructure to move data from A to B. You'll be expected to own architecture decisions independently and work with limited day-to-day guidance.

Our pipeline processes billions of records, so this is not a role for someone who has only worked with SQL/Spark at small-to-moderate scale. Query and job design choices here have real cost and runtime consequences, and inefficient code fails or times out in ways it wouldn't on a smaller dataset.

Responsibilities

What You'll Work On

• Large-scale data processing and ETL
• Data integration and normalization across diverse data sources
• Data quality and validation
• Querying and processing performance
• Cloud-based data infrastructure
• Data analysis and scoring systems
• Automated testing and production reliability
• Machine learning and statistical approaches to data quality
• Emerging AI, LLM, and agentic technologies

The specific problems will evolve as our product and data platform grow. We are looking for someone who can learn the system, understand the underlying data, and determine the best technical approach rather than simply following a predefined implementation pattern.

Education & Experience

• Minimum 4 year bachelors degree required from a reputable school with a concentration in computer science, engineering, mathematics, statistics or a related field of study
• 5+ years of professional data engineering experience, with demonstrated ownership of projects end-to-end (design through production)
• Comfortable making and defending architecture decisions with minimal oversight
• Comfortable owning a project end-to-end: from analysis/planning through implementation and testing
• Able to work with dependencies on other teams' data deliverables and adjust scope accordingly
• Clear communicator who can document architecture decisions and tradeoffs

Data Engineering

• Strong Python and SQL skills
• Significant hands-on experience with Apache Spark or a comparable distributed data processing technology
• Experience designing data processing solutions for large datasets where performance, scalability, and cost are important considerations
• Experience working with AWS cloud-based data platforms
• Experience building and maintaining production ETL or data processing pipelines
• Experience with automated testing and validation of data processing code

Analytical & Data Reasoning

• Strong ability to investigate unfamiliar data and identify the source of data quality or processing problems
• Experience using data analysis to evaluate business or product outcomes and guide engineering decisions
• Ability to reason about the accuracy and reliability of data rather than assuming that existing data or business rules are correct
• Experience evaluating changes using historical or production data before deploying them
• Ability to translate ambiguous data problems into measurable requirements and testable solutions
• Strong attention to detail when validating changes to production data

Systems & Architecture

• Experience working with multi-stage data pipelines and understanding dependencies across systems
• Experience with workflow orchestration or comparable distributed processing systems
• Ability to troubleshoot production systems across multiple services and identify where failures or unexpected results originate
• Experience working with data from multiple sources and developing consistent, reliable datasets
• Ability to understand and document complex data flows and system behavior