Lead Data Engineer
Quantitative Trading Firm · Mumbai, India ·
- Seniority
- Lead
- Category
- Data engineering
Quantitative Trading Firm · Mumbai, India ·
Lead a data engineering team at a quantitative/proprietary trading firm in Mumbai, owning the end-to-end data delivery process — procurement, pipeline design and deployment, and database insertion — and building tools that simplify data querying for quant developers. Core stack: Python, Bash, SQL, Linux, and ClickHouseDB.
Experience Level: 5 - 7+ Years Preferred Industry: High-Frequency Trading (HFT) or Hedge Funds Key Responsibilities: Handle the entire data delivery process, including data procurement, vendor discussions, process setup, pipeline design and deployment (with logging and alerts), and database insertion, optimizing for minimal data delivery cycles. Build applications that simplify data querying for quantitative developers. Work with multiple datasets to deliver standardized data outputs. Break down large, abstract tasks into manageable deliverables within set deadlines. Identify business requirements, communicate them effectively to end users, and clarify technical details as needed. Distinguish between user needs and wants to prioritize effectively. Proactively explore and propose new ideas. Quickly assume responsibilities from senior staff to support the team. Adhere to safe data handling practices. Essential Skills: Strong command of Python, Bash, and SQL in a Linux environment. Understanding of low-level system design for efficient data delivery applications. Ability to explain technical concepts to non-technical stakeholders. Hands-on experience with ClickHouseDB and its integrations with Python, Bash, and C++. Familiarity with Jira and Agile methodologies. Proficiency with Git and version control workflows. Comfortable developing over SSH. Knowledge of market, alternative, and supporting datasets with strong technical comprehension. Experience leading a team of at least 4 people and delivering robust data solutions. Ideal Skills: Experience handling 1+ TB of daily data processing. Ability to build highly efficient and resource-optimized data pipelines. Up to date with hardware and software trends relevant to data engineering. Ability to write efficient and maintainable code without relying on LLMs. Experience working in fast-paced and high-availability proprietary trading environments. Nice-to-Have Skills: Experience with serialization, lock-free queues, and multithreading in C++. Knowledge of multiprocessing and asynchronous programming in Python. Familiarity with various data processing frameworks. Experience with Azure data services. Needful Qualities: Well-organized, detail-oriented; able to multi-task and keep track of various deadlines in a fast-paced, high-demand environment. Highest level of integrity, empathy, and character. Mature and thoughtful, with the ability to operate in a collaborative, team-oriented culture. Why Join Us? Be part of a fast-growing, innovative firm at the forefront of financial technology. Work on challenging, high-impact projects with opportunities for rapid learning and career growth. Competitive compensation package with performance-based incentives. A dynamic work environment with exposure to cutting-edge technologies in trading.