Data Engineer
TalentOla · Mississauga, Canada ·
- Work mode
- Hybrid
- Employment
- Full time
- Category
- Data engineering
- Experience
- 9+ years
Job Title: Data Engineer/ Python Developer
Experience: Minimum 9 years
Employment: Fulltime, 5 days onsite
Mandatory Skills: Python, Spark, Pyspark
Responsibilities
- Design, develop, and maintain software solutions using Python.
- Develop Python and pyspark programs for data analysis. Good working experience with Python to develop a custom framework for generating rules (similar to a rules engine).
- Develop Python code to gather data from HBase and design a solution to implement using Pyspark. Apache Spark DataFrames/RDDs were used to apply business transformations and utilize Hive Context objects to perform read/write operations.
- Develop back‑end services, APIs, and integrate databases.
- Optimize applications for performance, security, and maintainability.
- Troubleshoot and resolve software defects and issues.