software engineer (web scraping / data acquisition)

Описание

NIQ is a consumer intelligence company that analyzes consumer buying behavior and identifies growth opportunities. Its platforms combine retail data and advanced analytics to deliver consumer insights. NIQ operates in more than 100 markets and covers over 90% of the world’s population.

Задачи

• Design and build distributed scraping architectures that scale across thousands of domains
• Reverse engineer APIs and mobile apps using tools such as Frida, mitmproxy, Charles Proxy, and Burp
• Detect anti-bot measures, including JavaScript checks and browser behavior detection, and build crawlers that adapt to appear more human-like
• Lead technical innovation in scraping, identify edge cases, prototype new approaches, and share knowledge with the team
• Set up anomaly detection and monitoring to identify blockers or failures early

Требования

• 5+ Years of experience with Python and large-scale scraping projects
• Deep understanding of anti-bot technologies and how to bypass them, including fingerprinting, WAFs, and JavaScript challenges
• Experience with Scrapy, Requests, httpx, Selenium, Playwright, or custom headless browser drivers
• Skilled in handling residential, datacenter, and rotating proxies, session management, cookie injection, and TLS tweaking
• Strong debugging skills across browser, HTTP traffic, and device-level interactions
• Independent problem-solving skills and enjoyment of tackling unsolved problems
• Good knowledge of MongoDB, InfluxDB, and Redis, including their operation and optimization
• Будет плюсом: familiarity with Parquet or Avro, real-time stream processing, real-device simulation for mobile scraping, distributed task queues such as Celery or Kafka, scraping high-security platforms or mobile-first environments, Linux or cloud environments such as AWS or GCP, basic Kubernetes, Cloud Run, and Cloud Functions knowledge for deploying and orchestrating scraping workloads

Условия

• Flexible working hours
• Flexible working environment
• Volunteer time off
• LinkedIn Learning
• Employee Assistance Program (EAP)