Dev-ops Engineer (3+ years).
We are seeking a DevOps Engineer with 3+ years of experience to support and operate OpenStack-based private cloud with Docker and Kubernetes platforms. This role focuses on maintaining the data platforms to ensure stable data handling by the Data Enginners, ML Engineers and Data scientists, automating routine tasks, maintaining CI/CD pipelines, and supporting durable, reliable and secure infrastructure operations.
Key Responsibilities
Working knowledge of big data related Dev ops tools, Big data open-source technologies on On-premises and big data platforms and tools like Airflow, Hadoop, Tableau, DBT dagster, kafka, MINio, Spark, Flink, Trino, Kubeflow.Will be completely working on on-premises. Knowledge of cloud platform will be added advantage.
Experience working on Linux based infrastructure
Excellent understanding of Python, SQL, Bash Shell, and Java
Awareness of critical concepts in DevOps
Develop software to integrate with internal back-end systems
Deploy updates and fixes, and provide Level 2 technical support
Build tools to reduce occurrence of errors and improve developer, customer experience
Perform root cause analysis of production errors and resolve technical issues
Develop scripts to automate visualization
Design procedures for system troubleshooting and maintenance.
Deep Understanding of Network Architecture and Cluster Setup like RACK, Spine Switch, leaf Switch, etc.
Improve CI/CD tooling.
Implement and improve monitoring and alerting.
High knowledge on data security and compliance like MIT kerberos, LDAP, IAM, etc.,.
Good Understanding of Container Technologies like Docker and Kubernetes.
Support the deployment, operation, and maintenance of OpenStack infrastructure under senior guidance.
Assist in managing Kubernetes workloads and Docker-based applications.
Help maintain CI/CD pipelines using Jenkins, GitHub Actions, or similar tools.
Support Infrastructure as Code (IaC) using Terraform and automation using Ansible, Python, or Shell.
Monitor systems and clusters using ELK, PLG (Prometheus, Loki, Grafana), or similar observability tools and get into action on need, providing the resolution.
Responsible for making the Big Data platforms up 24X7.
Should be well versed in pub-sub systems, Data and ML pipelines, Real time streaming and batch data processing.
Assist with troubleshooting Linux systems and infrastructure issues in production and non-prod environments.
Follow security best practices including IAM, RBAC, SSL/TLS, and least-privilege access.
Collaborate with senior engineers, follow technical direction, and continuously learning/documentation of DevOps implementations and best practices.