This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Databricks Data Engineer | Senior based in Brazil.
This is an opportunity for a senior data professional to design, migrate, and evolve modern data engineering solutions using Databricks. You will build scalable and reliable pipelines that transform raw data into high-quality, business-ready assets. The role involves working across Spark, PySpark, SQL, Delta Lake, and Lakehouse architectures. You will contribute to data platform modernization initiatives, including migrations and the implementation of Medallion architectures. Your work will directly support data-driven decision-making by translating business requirements into robust technical solutions. The environment emphasizes performance, reliability, monitoring, automation, and strong engineering practices. This is a strong fit for someone who enjoys solving complex data challenges and working with modern cloud data technologies.
Accountabilities
- Design, build, migrate, and evolve data engineering solutions within Databricks environments.
- Develop and maintain scalable, reliable, and high-performance data pipelines.
- Implement data ingestion, transformation, and delivery processes using Apache Spark and PySpark.
- Design and implement Data Lake and Data Lakehouse architectures using the Medallion model across Bronze, Silver, and Gold layers.
- Translate business rules and requirements into efficient data processes and technical solutions.
- Participate in data platform migration and modernization projects.
- Configure and manage Databricks Jobs, Workflows, orchestration, and pipeline monitoring.
- Implement data solutions using Delta Lake, Delta Tables, Unity Catalog, and Lakeflow Declarative Pipelines.
- Support data ingestion initiatives using Lakeflow Connect and related technologies.
- Ensure pipeline quality, performance, observability, reliability, and operational stability.
- Apply version control, CI/CD, and software engineering best practices throughout the development and deployment lifecycle.
Requirements
- Strong hands-on experience with PySpark, Apache Spark, and SQL.
- Practical experience with Databricks, including Delta Lake, Delta Tables, Unity Catalog, and Lakeflow Declarative Pipelines.
- Experience with Lakeflow Connect for data ingestion is desirable.
- Experience with Databricks Jobs, Workflows, orchestration, and pipeline monitoring.
- Solid understanding of Lakehouse architecture and the Medallion model using Bronze, Silver, and Gold layers.
- Previous experience migrating data platforms or delivering data engineering projects using Databricks.
- Knowledge of Git, CI/CD, and modern development and deployment practices.
- Experience with or knowledge of technologies such as Airflow, Kafka, dbt, AWS Glue, and BigQuery.
- Experience working with SQL and NoSQL databases, including technologies such as PostgreSQL, MongoDB, or Cassandra.
- Strong analytical and problem-solving abilities, with the capacity to translate business requirements into scalable data solutions.
- Ability to work collaboratively with technical and business stakeholders in data modernization initiatives.
- Experience with accounting or financial projects is a plus.
- Databricks Certified Data Engineer Associate certification is considered an advantage.
Benefits
- Remote work model.
- Full-time employment.
- Opportunity to work with modern data engineering technologies and Databricks-based Lakehouse architectures.
- Exposure to large-scale data migration and modernization initiatives.
- Opportunity to work with Spark, PySpark, Delta Lake, Unity Catalog, Lakeflow, and other cloud data technologies.
- Professional development in a technology-driven environment focused on AI and modern digital platforms.
- Opportunity to collaborate with experienced technology professionals on complex data engineering challenges.
- Accessibility and inclusion: opportunities are also open to people with disabilities.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1