Senior Data Engineer
Valce Talent Solutions ·
- Seniority
- Senior
- Category
- Data engineering
Valce Talent Solutions ·
Amgen · India - Hyderabad
Dataeconomy · Hyderabad, India
GSG-GESTIÓN DE SERVICIOS GLOBALES S.A.S · Bogotá, Colombia
gatesfoundation · Seattle, WA
Senior Data Engineer designing and maintaining scalable batch and real-time data pipelines for enterprise credit risk and lending analytics. Day-to-day work centers on Databricks, Spark/PySpark, SQL, Python, Airflow, Delta Lake and cloud-native (Azure) lakehouse tooling, plus data quality, CI/CD and mentoring junior engineers. Fully remote; advanced English required.
Senior Data Engineer
Role
As a Senior Data Engineer, you will:
Design, develop, test, and maintain scalable data pipelines and data products supporting Enterprise Credit Risk initiatives.
Build and optimize batch and real-time data processing solutions using modern cloud and lakehouse technologies.
Partner with Risk, Product, Data Science, and Engineering teams to understand business requirements and translate them into technical solutions.
Develop reusable frameworks, components, and engineering patterns that improve development efficiency and platform consistency.
Implement data quality controls, monitoring, alerting, and observability capabilities to ensure data reliability and trustworthiness.
Support the migration and modernization of existing data assets and workloads into Databricks and cloud-native platforms.
Build and maintain curated datasets that support lending decisioning, portfolio risk management, and advanced analytics use cases.
Participate in architecture discussions and contribute technical recommendations for platform enhancements.
Troubleshoot production issues and drive root cause analysis to improve platform reliability and performance.
Follow engineering best practices for code quality, testing, deployment automation, security, and operational excellence.
Mentor junior engineers through code reviews, technical guidance, and knowledge sharing.
Continuously evaluate opportunities to improve scalability, performance, cost efficiency, and maintainability of data platforms.
All About You
Essential Skills & Experience
Strong experience developing and maintaining enterprise-scale data engineering solutions.
Hands-on experience with Databricks, Apache Spark, PySpark, Hadoop, SQL, and Python.
Experience building ETL/ELT pipelines and large-scale data processing applications.
Experience working with cloud-based data platforms and storage technologies.
Strong understanding of data modeling concepts and analytical data structures.
Experience implementing automated testing, monitoring, and data quality practices.
Experience with Data formats ( Parquet, Avro, ORC )
Knowledge of CI/CD pipelines, Git-based development workflows, and DevOps principles.
Experience with Workflow orchestration Tools like Airflow
Strong analytical and problem-solving skills with the ability to work independently on complex technical challenges.
Effective communication skills with the ability to collaborate across technical and business teams.
Experience mentoring or supporting less experienced engineers.
Knowledge of Java Based application development is a huge Plus.
Preferred Qualifications
Bachelor's degree in Computer Science, Engineering, Information Systems, or related STEM field or alternative minimum of 5 years of experience in a related field.
Technical Skills
Experience with several of the following technologies:
Databricks
Apache Spark / PySpark
AirFlow
SQL
Python
Delta Lake
Azure Data Services
Data Factory or equivalent orchestration tools
Kafka or streaming platforms
GitHub
CI/CD automation
Data Quality and Observability tools
ADVANCED ENGLISH
REMOTE
GSSTech Group · Dubai, UAE