Senior Data Engineer
Quest Global · Hyderabad, India ·
- Seniority
- Senior
- Category
- Data engineering
- Experience
- 5+ years
Quest Global · Hyderabad, India ·
RDSolutions India · Hyderabad, India
Procorp Systems · Hyderabad, India
OEC · Oggiyamduraipakkam, TN, IN
ecsme · Telangana, Hyderabad, India
Designs, builds, and maintains large-scale data pipelines, ETL processes, and data warehouse architectures, handling data migration and integration from multiple sources. Core stack is SQL, Python, Hadoop/Spark, and AWS services including Redshift, S3, Glue, EMR, Kinesis, and Lambda.
Job Title: Data Engineer
Location: Hyderabad
Experience: 5+Years
Job Summary:
We are looking for a skilled and experienced Data Engineer with over 5 years of experience in data engineering and data migration projects. The ideal candidate should possess strong expertise in SQL, Python, data modeling, data warehousing, and ETL pipeline development. Experience with big data tools like Hadoop and Spark, along with AWS services such as Redshift, S3, Glue, EMR, and Lambda, is essential. This role provides an excellent opportunity to work on large-scale data solutions, enabling data-driven decision-making and operational excellence.
Key Responsibilities:
• Design, build, and maintain scalable data pipelines and ETL processes.
• Develop and optimize data models and data warehouse architectures.
• Implement and manage big data technologies and cloud-based data solutions.
• Perform data migration, data transformation, and integration from multiple sources.
• Collaborate with data scientists, analysts, and business teams to understand data needs and deliver solutions.
• Ensure data quality, consistency, and security across all data pipelines and storage systems.
• Optimize performance and manage cost-efficient AWS cloud resources.
Basic Qualifications:
• Master's degree in computer science, Engineering, Analytics, Mathematics, Statistics, IT, or equivalent.
• 5+ years of experience in Data Engineering and data migration projects.
• Proficient in SQL and Python for data processing and analysis.
• Strong experience in data modeling, data warehousing, and building data pipelines.
• Hands-on experience with big data technologies like Hadoop, Spark, etc.
• Expertise in AWS services including Redshift, S3, AWS Glue, EMR, Kinesis, Firehose, Lambda, and IAM.
• Understanding of ETL development best practices and principles.
Preferred Qualifications:
• Knowledge of data security and data privacy best practices.
• Experience with DevOps and CI/CD practices related to data workflows.
• Familiarity with data lake architectures and real-time data streaming.
• Strong problem-solving abilities and attention to detail.
• Excellent verbal and written communication skills.
• Ability to work independently and in a team-oriented environment.
Good to Have:
• Experience with orchestration tools like Airflow or Step Functions.
• Exposure to BI/Visualization tools like QuickSight, Tableau, or Power BI.
• Understanding of data governance and compliance standards.
Calsoft · Maharashtra, India