data engineer in data platforms

Описание: Simple Machines is a global, independent technology consultancy that designs and builds modern data platforms, intelligent systems, and bespoke software across data engineering, software engineering, and AI. It helps enterprises, scale-ups, and government turn complex data into products, platforms, and actionable decisions.

Задачи

• Own the end-to-end architecture of modern, cloud-native data platforms
• Design scalable data ecosystems using data mesh, data products, and data contracts
• Make architectural decisions across ingestion, storage, processing, and access layers
• Ensure platforms are secure, compliant, and production-grade by design
• Design and deliver cloud-native data platforms using Databricks, Snowflake, AWS, and GCP
• Integrate with client systems to enable scalable, consumer-oriented data access
• Build and optimise batch and real-time pipelines
• Work with streaming and event-driven technologies such as Kafka, Flink, Kinesis, and Pub/Sub
• Orchestrate workflows using Airflow, Dataflow, and Glue
• Process and transform large datasets using Spark and Flink
• Design production-ready systems
• Work across relational, NoSQL, and analytical data stores
• Optimise storage formats and access patterns
• Implement secure, compliant data solutions with security by design
• Embed governance while maintaining developer velocity
• Work directly with clients to understand problems and shape solutions
• Translate business needs into pragmatic engineering decisions
• Act as a trusted technical advisor
• Set engineering standards, patterns, and best practices across teams
• Review designs and code, providing technical direction and mentorship
• Improve data quality, testing, observability, and operational excellence

Требования

• Strong Python and SQL skills
• Deep experience with Spark and modern data platforms such as Databricks and Snowflake
• Solid understanding of cloud data services in AWS or GCP
• Demonstrated ownership of large-scale data platform architectures
• Strong data modelling and architectural decision-making skills
• Ability to balance performance, cost, and complexity trade-offs
• Experience building and operating large-scale data pipelines in production
• Experience with multiple storage technologies and formats
• Infrastructure-as-code experience with Terraform or Pulumi
• Experience with CI/CD pipelines using tools such as GitHub Actions or ArgoCD
• Experience with data testing and quality frameworks such as dbt, Great Expectations, or Soda
• Experience in consulting or professional services environments
• Strong consulting instincts and ability to challenge assumptions and guide clients toward better outcomes
• Ability to mentor senior engineers and influence technical culture

Условия

No conditions specified