Clearwater is the only company combining deep healthcare security and compliance expertise with MSSP capabilities, managed cloud services, consulting and assessments, and compliance software. We’re here to help organizations become more secure, compliant, and resilient.
Why and What we expect:
For the purpose of Clearwater engineering team extension we are looking for a Data engineer, who is excited to face with different data security challenges working in a "Dream" team of experiences engineers of the enterprise US company.
Responsibilities:
- Develop and maintain a multi-layered data pipeline to ingest, transform, and analyze raw JSON data into optimized Parquet files.
- Utilize PySpark to execute large-scale data transformations and analytical queries across a distributed cluster, ensuring high performance and scalability.
- Integrate AWS services (S3, Lambda, Glue, EMR, etc.) into the pipeline to manage data ingestion, processing, and archival effectively. Optimize cloud resource usage and manage costs.
- Implement robust ETL processes that validate, clean, deduplicate, and enrich data. Develop strategies for efficiently offloading raw data to cost-effective storage while maintaining accessibility for reprocessing.
- Commit code to GIT (tested on development stage and ready for test stage)
- Communicate with customer:
- daily standup meetings with project team;
- meetings with customer on demand.
Process:
- Follow project work schedule: 10 am- 7 pm , 1-2 hour for break/lunch
- Give a feedback/propositions regarding new ideas and possible improvements of product or engineering process to PM in a written form.
- Passionate: Always sharpen the technology skills
- Agile : Learn and adapt to new technologies
- Self-motivated: Take initiative to learn the code and industry and dig into new challenges
- Quality focused: Excellence is in the details
- Flexibility: Shift focus with changing business needs
- High attention to details and accuracy;
- Team player and experience of working with a US client;
- Proactive approach of work in a mindset;
- Creative: Look for elegant solutions that are not always the obvious ones
- Patience and tolerance, calmness and sense of humor.
Professionalskills:
- 5+years ofexperience indata engineering using Python, knowledge ofJava and Scala would beaplus;
- Strong analytical and problem-solving skills;
- Proficiency inSQL (PostgreSQL);
- Experience with PySpark and Spark;
- Experience with AWS services suchas: S3, Lambda, Glue, EMR, etc.;
- Familiarity with Kubernetes isaplus;
- Profiling, debugging and troubleshooting experience;
- Experience with Git;
- Knowledge ofLinux based platforms;
Conditions of work:
Corporate laptop for remote work and fully equipped desk including snacks and drinks in our engineering office on 38th floor of Warsaw Spire building with breathtaking panoramic views to city center.
Super Bonuses:
- Opportunity to become a part of our DREAM TEAM, enjoy the atmosphere and grow together with the company building something very huge.
- Participation in the most revolutionary projects in AI and Big Data technology niches;
- Competitive salary, personal bonuses for excellent delivery and permanent professional growth.
- Paid sick leave and vacation.
- Corporate education programs, certification, and participation in the corporate knowledge sharing process.