Data Engineer
Consortia Group · Norway ·
- Employment
- Full time
- Category
- Data engineering
- Experience
- 3+ years
DataEngineer
London|Hybrid|Full-time
AreyouaskilledDataEngineereagertoworkwithhigh-volumedatasetsandcutting-edgetechnologies?We’relookingforadata-drivenprofessionaltodevelopandoptimiseascalabledataplatformthatsupportsadvancedanalyticsandquantitativeresearch.You’llplayakeyroleindesigningrobustdatapipelines,integratingreal-timefeeds,andensuringtheseamlessflowofstructuredandunstructureddata.
Inthisrole,you'llberesponsibleforbuildingandmaintaininghigh-performancedatapipelinesthatprocesscomplexandlarge-scaledatasets.You'llworkcloselywithanalystsandresearcherstounderstandtheirdatarequirements,optimisingreal-timedataingestionbyintegratingexternalAPIsandalternativedatasources.Youwillfocusonenhancingthedataarchitecturetoimproveaccessibility,efficiency,andsecuritywhileautomatingETLworkflowstostreamlinedatamodellingandanalysis.You’llalsosupportevent-drivensystemsbyimplementingtechnologieslikeKafkaandProtobuf,ensuringseamlessdatastreamingandprocessing.Akeyaspectofyourrolewillbeimplementingrigorousqualityassurancemeasures,ensuringdataintegrityandconsistencyacrossallinputsandoutputs.Additionally,you'llcollaboratewithteamstosupportmachinelearningapplications,facilitatingdatapreparationformodeltraininganddeployment.
WhatYou’llNeed
- 3+yearsofexperienceinadataengineering
- StrongproficiencyinPythonandRust
- Hands-onexperiencewithAWScloud,Linux,andDocker.
- Expertiseindistributedcomputingframeworks(Ray,Dask,orApacheArrow).
- Stronganalyticalskills,withtheabilitytohandlelarge-scale,high-frequencydata
- .SolidunderstandingofETLpipelines,datamodelling,anddatabaseoptimisation.
- Experienceworkingwithreal-timestreamingtechnologies
Thisisafantasticopportunitytobepartofafast-moving,data-drivenenvironmentwhereyourtechnicalexpertisewillhavearealimpact.You’llworkwithcutting-edgetools,collaboratewithtop-tierprofessionals,andcontributetotheefficiencyandinnovationofagrowingdatateam.