Talent Destination is hiring a Data Engineer for a 6-month contract (listed as remote) to build, optimize, and debug batch and real-time data pipelines at scale. Core stack is Databricks on AWS with Spark, Airflow, Terraform, strong Python, and CI/CD tooling like Git and Jenkins.
Strong hands-on experience with Databricks, ideally on AWS (inc. SQL, Spark, Airflow, Terraform)·
Strong Python experience, including deep knowledge of the Python data ecosystem; hands-on in orchestration tooling.
Hands-on experience with implementing CI/CD, using Git, Jenkins, or similar technologies.
Hands-on experience in all phases of data modelling from conceptualization to database optimization
Solid hands-on experience implementing batch and real-time data integration frameworks including assessing performance, debugging, and fine-tuning those systems
Demonstrated ability to perform the engineering necessary to acquire, ingest, cleanse, integrate, and structure massive volumes of data from multiple sources.