Staff Data Engineer

Role and Responsibilities:

• Participate in the entire data pipeline lifecycle, focusing on building data lakes, data warehouses and related processes
• Contribute to the design and documentation of our data lake platform
• Partner with Sr staff on the development and maintenance of data governance policies and procedures that ensure data quality, accuracy, consistency, and privacy.
• Mentor other data engineers, providing technical guidance, and ensuring that team members have the necessary skills and resources to perform their duties.
• Conduct code and design reviews with junior and peer developers
• Write clean code to develop data pipelines and necessary infrastructure as code
• Aid in setting coding standards for team members to follow
• Partner with cross functional teams to understand requirements and effectively design appropriate solutions.
• Monitor and troubleshoot performance, scalability, and security issues, making improvements as needed
• Aid in determining and implementing KPIs for performance and stability of platform
• Participate in the review of new technologies and tools for our data teams
• Partner with engineering and other cross functional teams to collect and implement technical and design requirements
• Communicate effectively with technical and non-technical team members to ensure alignment and successful execution of projects.
• Participate in cross-functional initiatives, promoting a culture of data-driven decision-making and continuous improvement.
• Participate in an on call rotation.
  Job Requirement :
• 8 years of related experience; or 6 years and an advanced degree
• Bachelors degree in engineering, computer science, math, data engineering or another related field or equivalent work experience
• Experience in the development of modern data architecture, analytics, data governance, AI/ML, or related areas.
• Strong Experience with AWS big data technologies including EMR, Glue, S3, EKS, Lambda, Athena, RDS, MKS/Kafka/Kinesis
• Expertise with relational and NoSQL databases such as PostgresQL, Cassandra, DynamoDB, MongoDb etc and data modeling principles
• Knowledge of data pipeline tools and workflow engines, e.g. Apache Airflow and Spark.
• Experience with DevOps, including CI/CD pipelines, containerized deployment/Kubernetes, and infrastructure-as-code/AWS Cloud Formation/Terraform.
• Successfully contributes to the entire development lifecycle of teams products
• Exposure to secure coding practices, access controls, authentication, and audit. AWS Lake formation preferred.
• Strong experience with SQL (SQL Server, Postgres) and Python, Scala
• Identify opportunities for process improvement as well as efficiency in solutions
• Demonstrated ability to mentor junior team members
• Excellent analytical and time management skills, with a proven ability to deliver value independently
• Strong written and verbal communication skills, with demonstrated experience providing technical input to technical and non-technical stakeholders
• Ability to participate in on-call rotation

Preferred:

• FinTech industry experience
• Understanding of MPP Data Warehouse (Redshift, Greenplum)
• Experience with Graph database