Senior QA Testing Specialist -Data Engineer

Role Description

Role Objectives

Data Quality & Validation:

• Perform end‑to‑end data validation, including source‑to‑target mapping, reconciliation, and completeness checks. Validate data transformations across Databricks Medallion Architecture (Bronze, Silver, Gold).
• Ensure data quality standards are met prior to production releases.

Test Strategy & Execution:

• Participate in requirements analysis and solution design reviews.
• Define test scope, scenarios, risk‑based coverage, and overall test strategies.
• Design and execute test cases for complex ETL pipelines and Perform regression testing and manage test data conditions.

Data Validation & Automation:

• Write and execute advanced SQL queries (CTEs, window functions, complex joins).
• Develop Python validation scripts using PySpark and Pandas to support and enhance data test automation efforts where applicable.
• Monitor Databricks jobs and workflow executions. Investigate pipeline failures, data inconsistencies, and quality issues. Analyze defects both functionally and technically prior to assignment.

Defect Management & Collaboration:

• Log, track, and manage defects using JIRA, ensuring high‑quality defect documentation.
• Collaborate closely with Data Engineers, Architects, and Business teams to resolve issues.
• Provide clear and timely communication on quality risks, test progress, and release readiness and Escalate issues through defined governance and quality escalation channels.

Qualifications and Skills

• 8+ years of experience in Quality Assurance testing or relevant Data Engineering development roles.
• Demonstrated expertise in Advanced SQL, including CTEs, window functions, and complex joins, for large‑scale data validation.
• 6+ years of experience in test management strategy; experience in the securities or financial services industry
• 3+ years of experience in designing solutions in process flow, business logic and UI.
• Strong experience within the Azure data ecosystem, with hands‑on expertise in Databricks notebooks and cluster management.
• Experience building or supporting data test automation frameworks, with exposure to CI/CD pipelines and Git‑based version control.
• Hands-on manual and automated test development and design experience leading medium to large scales or enterprise-wide projects and production implementations.
• Strong command of Python, particularly PySpark and Pandas, for comprehensive data validation and analysis.
• Thorough understanding of ETL processes, data pipelines, and data warehouse architecture, with proven experience in ETL/Data Warehouse testing, including end source‑to‑target validation.
• Prior experience working directly with business users and finance stakeholders, including exposure to Oracle or similar enterprise financial platforms.
• Strong analytical, critical‑thinking, and problem‑solving skills.
• Excellent written and verbal communication skills, with the ability to articulate technical concepts clearly.
• Ability to work independently while collaborating effectively across distributed and global teams.

Measures of Success

• High level of confidence in production data quality, reflected by reduced defect leakage post‑release.
• Strong and consistent test coverage across complex, multi‑layer data platforms.
• Effective collaboration with engineering, product, and business teams throughout the delivery lifecycle.
• Proactive identification and mitigation of data quality risks and issues.
• Consistent, on‑time delivery of test outcomes without compromising quality standards.