Introduction:
We at Entiovi Technologies provide digital transformation using new-age intelligent technologies for more than 9 years. We have clients located primarily in the US and Europe that are served by our Dedicated teams. This job is part of our expansion in India.
Location: Remote
About the role:
The Senior Data Engineer in our AI & Data team will be responsible for designing and building the data structures and pipelines our AI Engineers rely on across Azure, Snowflake, Databricks, and Lakebase (our managed Postgres / OLTP layer). The primary mission of this role is to enable the AI Engineering team by translating the needs of machine-learning and computer-vision workflows into reliable, well-modelled, and cost-effective data foundations.
Tasks include setting up new data pipelines and transformations, ingesting structured and unstructured data into the data lake and warehouse, monitoring the performance and cost-effectiveness of existing data jobs, and docking machine-learning processes into the existing data landscape. The Data Engineer works hand in hand with AI Engineers and is the go-to person for making trusted data available for models, products, and analytics.
Main Responsibilities
- As part of the AI & Data team, design and build the data structures, schemas, and models that AI Engineers depend on for training, feature engineering, and inference.
- Strong experience in Data Architecture.
- Strong knowledge of Git repositories.
- Develop and orchestrate scalable data pipelines on Databricks (Spark, Delta Lake) and load curated, analytics-ready data into Snowflake.
- Own data ingestion, transformation (ELT/ETL), and storage across the Azure cloud (e.g. ADLS, Data Factory, Event Hubs / Synapse), including structured, semi-structured, and unstructured data such as text, images, and video.
- Dock machine-learning and computer-vision models into the data pipelines and design the data flow that feeds and consumes those AI services.
- Sync curated lakehouse data into Lakebase (managed Postgres) for low-latency serving, manage change-data-capture back into Delta tables, and support online feature stores and agent state for AI Engineers.
- Strong API knowledge and experience.
- Build and maintain API integrations and automated data ingestion from internal systems and external third-party sources.
- Monitor pipeline performance, reliability, and cost; troubleshoot failed jobs and optimize Snowflake and Databricks workloads.
- Implement data quality, validation, and lineage, and document the data dictionary and ETL processes.
- Partner with AI Engineers and stakeholders to translate model and business requirements into extensions of the data platform.
Skills, Qualifications & Education
- Bachelor’s degree in Computer Science, Data Engineering, or a related field.
- At least 4 years of work experience in data engineering or a similar data-focused role.
- Hands-on production experience with Databricks (Apache Spark, Delta Lake, notebooks, workflows).
- Hands-on production experience with Snowflake (data modelling, performance tuning, access control, cost management).
- Solid experience with Microsoft Azure data services (e.g. ADLS, Data Factory, Event Hubs / Synapse).
- Experience with PostgreSQL and OLTP databases; familiarity with Lakebase (Databricks-managed Postgres) is a strong plus.
- Working knowledge of JavaScript / TypeScript, used for data APIs, microservices, or app-facing integrations.
- Strong expertise in SQL (will be tested during the recruitment process).
- Robust Python literacy, especially for data handling and pipeline development.
- Comfortable working with both structured and unstructured data; does not shy away from troubleshooting failed ETL processes or API integrations.
- Outstanding data-structure and data-modelling design skills.
- Working knowledge of machine-learning, NLP, or computer-vision workflows is a plus.
- Experience with dbt, Airflow, or Databricks Workflows, and with CI/CD and infrastructure-as-code, is a plus.
- Strong ability to translate ideas between technical and non-technical audiences.
- Curious, collaborative, self-motivated, and organized; able to run multiple projects against tight deadlines.