Data Operations Support Specialist

Job Description:

A Data Operations Support Specialist plays a pivotal role in ensuring the smooth and efficient operation of data processing systems, encompassing Google Dataflow, Google Cloud Big Query, Google Composer, and IBM DataStage. This multifaceted role involves monitoring, troubleshooting, optimizing data pipelines, ensuring data quality, and providing technical support to data engineers and analysts to meet the organization's data processing needs.

Key Responsibilities:

• Monitor and manage data processing workflows in Google Dataflow and IBM DataStage.
• Oversee the performance, availability, and data integrity of Google Cloud Big Query datasets and queries.
• Manage Google Composer workflows for orchestrating data processing tasks.
• Implement data quality checks and validation routines within data pipelines to ensure data accuracy and consistency.
• Identify and address data quality issues promptly.
• Troubleshoot data pipeline issues, system failures, and performance bottlenecks across all platforms.
• Collaborate with data engineers and developers to resolve technical problems and enhance system reliability.
• Create and maintain comprehensive documentation of data pipelines, configurations, and operational procedures for all platforms.
• Identify opportunities to enhance system performance, reduce processing time, and optimize costs in Google Dataflow, IBM DataStage, and Google Cloud Big Query.
• Ensure data security by implementing access controls, encryption measures, and compliance with data privacy regulations.
• Collaborate with cross-functional teams, including data engineers, data analysts, and business stakeholders, to understand requirements and provide operational support.
• Train end-users and provide technical support to resolve issues related to data operations for all platforms.
• Assist in educating teams on best practices for data processing and orchestration across all systems.
• Develop and maintain scripts or automation tools to streamline repetitive tasks and enhance operational efficiency across platforms.

Qualifications:

• Bachelor's degree in computer science, information technology, or a related field.
• Proficiency in Google Dataflow, Google Cloud Big Query, Google Composer, and IBM DataStage.
• Extensive experience with data integration, ETL processes, and workflow orchestration across multiple platforms.
• Strong troubleshooting and problem-solving skills.
• Knowledge of SQL, data warehousing, and data modeling concepts.
• Familiarity with cloud computing and infrastructure, particularly in Google Cloud and IBM environments.
• In-depth understanding of data security, privacy, and compliance.
• Excellent communication and collaboration skills.
• Ability to work independently and in a team.