We are seeking a Lead Data Software Engineer to drive the migration of an existing data analytics platform, originally built on Azure resources such as DataFactory, Databricks, EventHub, and Cassandra, to a cloud-agnostic platform capable of on-premises deployment.
Responsibilities
- Implement streaming and batch Spark pipelines running on K8S
- Develop objects of the data generator framework
- Deploy and test solutions on local and development environments
- Enhance and refine the current actively evolving solution
- Investigate and resolve bugs
- Conduct unit testing to ensure code quality
- Participate in refinement, planning, and demo sessions
- Guide and mentor team members while syncing with the team lead
Requirements
- 5+ years of experience with Apache Spark, Databricks, and Kubernetes (K8S)
- Proficiency in Python
- Experience with Confluent/Apache Kafka
- Completed EPAM Data Engineering course or equivalent qualification
- Capability to quickly adapt and dive into the active implementation phase of a project
- Ability to work independently while coordinating with a team lead
- Strong communication skills and a proactive, hands-on, team-player attitude
- English proficiency at B2 level or higher
Nice to have
- Knowledge of Java
- Familiarity with Kafka Connect, KSQL, and Kafka Streams
- Background in Ansible and Argo CD
- Understanding of TDD
Benefits
CONTINUOUS UPSKILLING, LEARNING & DEVELOPMENT
- Diversity of tasks and projects
- Assessment center for objective review of competency level
- Personal development plan
- Mentoring programs and leadership development
- Certification and professional development support
- Access to learning platforms including more than 2,500 internal courses
- English courses taught by certified teachers
CORPORATE BENEFITS
- Extra leave days
- Referral bonuses
COMPENSATION PACKAGE
- Competitive compensation paid in USD
- Regular salary and performance reviews
MEDICAL & HEALTHCARE
- Private health insurance
- Well-being events
WORKING ENVIRONMENT
- Recreation areas and kitchens
- Tea, coffee and snacks
- Sports equipment and game consoles
- IT Equipment
- Microsoft’s Software Assurance Home Use Program (HUP)