About the role
Pay Range: £75k-£200k (salary + equity), based in London, UK.
We are hiring a Data Engineer to join Squid as one of our first engineers.
You will work closely with the founders and early engineering team to build the data foundations of our platform for grid operators. The role combines data engineering, software engineering, infrastructure and applied AI.
You will work with large, complex and often inconsistent technical datasets, building reliable systems to ingest, transform, validate, connect and serve them. This is a high-ownership role based in our central London office, with the opportunity to shape our data architecture, engineering practices and product from an early stage.
What you'll do
- Build production data pipelines for ingesting, transforming, validating and serving complex datasets.
- Design scalable data models, schemas and storage systems for structured, semi-structured and geospatial data.
- Build pipeline orchestration using directed acyclic graphs, or DAGs, to manage dependencies, retries, scheduling and observability.
- Develop systems for data quality testing, lineage, versioning, reconciliation and change detection.
- Create tools for combining datasets from different systems while preserving provenance and auditability.
- Build APIs and services that make processed data available to products, customers and AI systems.
- Develop workflows for batch processing, event-driven processing and long-running computational jobs.
- Create monitoring and alerting for pipeline failures, unexpected data changes and quality regressions.
- Work directly with customers and domain experts to understand source systems and data problems.
- Use modern AI development tools to accelerate data mapping, validation and pipeline development.
Qualifications
You do not need to meet every requirement. We are looking for strong data engineers who enjoy making difficult, messy datasets reliable and usable.
You may be a strong fit if you have:
- Professional experience building and maintaining production data pipelines.
- Strong Python and SQL skills, with experience working with Postgres or similar databases.
- Experience designing data models, schemas and transformation layers for complex datasets.
- Experience with directed acyclic graph-based orchestration tools such as Airflow, Dagster, Prefect or similar systems.
- Familiarity with data warehouses, object storage, queues, APIs and cloud infrastructure.
- Experience with data quality testing, lineage, observability, versioning or change-data capture.
- The ability to work with incomplete, inconsistent or poorly documented source data.
- Strong software engineering fundamentals, including testing, code review and maintainable system design.
- An interest in applied AI, data evaluation and building reliable data foundations for AI systems.
- High ownership, clear communication and a willingness to work directly with technical customers.
Experience with geospatial data, graph data, time-series data, scientific computing, energy data or other complex engineering datasets would be valuable, but is not required.