Deploy, manage, and support data pipelines in production environments.
Build and maintain scalable data processing solutions on AWS cloud platforms.
Automate testing, deployment, and monitoring of data pipelines through CI/CD practices.
Ensure reliability, performance, and operational stability of data platforms and workflows.
Collaborate closely with Data Engineers, Machine Learning Engineers, Software Engineers, Product Managers, and domain experts to deliver data-driven solutions.
Troubleshoot production issues, perform root cause analysis, and implement preventive measures.
Follow Agile development practices and contribute to continuous improvement initiatives.
Requirements
Minimum 5 years of experience deploying and managing data pipelines in production environments.
Proven experience working in cross-functional teams comprising Data Engineers, Machine Learning Engineers, Software Engineers, Product Managers, and business stakeholders.
Strong proficiency in SQL and relational database technologies.
Hands-on experience with at least one workflow orchestration tool such as Airflow or Dagster.
Strong Python programming skills, including experience with data processing libraries such as Pandas.
Experience with containerization technologies, particularly Docker.
Familiarity with CI/CD tools and practices, preferably Jenkins.
Experience working in Agile development environments.
Preferred Skills
Hands-on experience with AWS services such as RDS, EKS, EMR, and Redshift.
Experience with Snowflake data platforms.
Experience developing and deploying microservices using Flask and/or FastAPI.
Knowledge of big data technologies such as Spark, Hadoop, and Kafka.
Familiarity with SQLAlchemy and Alembic.
Experience supporting large-scale data platforms, analytics, or machine learning workloads.
Nice to Have
Exposure to MLOps, DataOps, or cloud-native data platform architectures.
Experience implementing infrastructure automation and observability solutions.
Knowledge of security and governance best practices within cloud-based data environments.