Python / Spark / AI Developer

Build Python & PySpark pipelines on Delta Lake

Replatform legacy T-SQL to Spark SQL

Develop tested, type-safe Python with CI quality gates

Build FastAPI REST APIs with authentication & job tracking

Validate migrated pipelines against legacy systems

Location:

Cape Town, Remote/Hybrid or In office candidate can decide

Job Type:

Permanent

Date:

2 days ago

Requirements

4+ years Python development

2+ years production Spark/PySpark

Strong Python 3.12, pytest, Pydantic & modern tooling

Spark SQL, DataFrames, performance tuning & distributed architecture

Docker, Git & CI/CD

Able to work independently in a spec-driven environment

Nice to Have

Observability Data Migration & Parity Testing Databricks Certification

Qualifications:

Bachelor’s Degree in Computer Science, Engineering or equivalent experience

Spark/Databricks certification advantageous