Data Engineer - Hadoop
Luxoft · Chennai, IN ·
- Category
- Data engineering
- Experience
- 4+ years
Luxoft is hiring a Data Engineer in Chennai to support an investment banking client's regulatory reporting initiative. Day to day: developing and optimizing SQL queries, scheduling and troubleshooting Control-M batch jobs, monitoring production pipelines, and processing/validating large datasets using the Hadoop ecosystem.
Project description
We've been engaged by an Investment Banking client to work on Regulatory Reporting initiative in the Capital Risk portfolio.
Responsibilities
- Develop, maintain, and optimize SQL queries for data extraction, transformation, validation, and analysis across large datasets.
- Design, monitor, and troubleshoot batch jobs and data processing workflows using job scheduling and monitoring tools, preferably Control-M.
- Manage daily production job schedules, monitor job execution, investigate failures, and ensure timely resolution to minimize operational impact.
- Work with Hadoop ecosystem technologies to process, analyze, and troubleshoot large volumes of structured and unstructured data.
- Perform data validation, reconciliation, and quality checks to ensure data accuracy, completeness, and consistency.
- Analyze data and operational issues, identify root causes, and implement appropriate solutions to improve reliability and performance.
SKILLS
Must have
- 4 to 6 years of relevant experience in Data & Analytics.
- Hands-on experience in SQL, Job Scheduling and Monitoring Tool preferably Control M
- Good understanding and working experience in Hadoop.
- Strong analytical and problem-solving skills.
- Ability to work independently and collaborate effectively with cross-functional teams.
Nice to have
• Experience of Agile/Scrum development methodologies is a plus • Be nice, respectful, able to work in a team • Willingness to learn