You will design and build batch and streaming data pipelines to feed AI, LLM, RAG systems, and Web3 products. You will manage ETL/ELT workflows with Airflow, ingest and transform real-time streams with Kafka, and create modular dbt models with clear documentation. You will write clean Python and optimized SQL to support analytics and downstream systems, and collaborate with AI and product teams to ensure data is accurate, timely, and usable.
Responsibilities
- Design and build batch and streaming data pipelines for AI, LLM, RAG systems, and Web3 products
- Manage ETL and ELT workflows using Apache Airflow
- Ingest and transform real-time data streams using Apache Kafka
- Build and maintain modular, testable dbt models with documentation
- Work with Data Warehouses to support analytics and product insights
- Collaborate with AI and product teams to provide structured reliable data
- Write clean efficient Python code and optimized SQL for scalable data processing
Requirements
- 3+ years of experience as a Data Engineer or similar role
- Strong SQL skills and data modeling experience
- Practical experience with Apache Airflow, Apache Kafka, and dbt
- Familiarity with PostgreSQL, MongoDB, or equivalent RDBMS/NoSQL solutions
- Solid Python programming skills for building data tools and automation
- Experience working with modern Data Warehouse platforms (e.g., Snowflake, BigQuery, Redshift)