Description
Manual & automation QA engineer
We're looking for a skilled QA engineer who's equally comfortable crafting detailed test plans by hand and building robust automation frameworks. You'll ensure that every feature we ship meets the highest quality bar — including our growing suite of AI-powered capabilities.
Our stack
Test automation: Playwright / Cypress
Unit & integration: Jest / Pytest
API testing: REST & GraphQL
Database: PostgreSQL
DevOps: CI/CD pipelines, GitHub Actions, AWS
AI: AI agents / LLMs (non-deterministic testing)
What you'll do
- Design, write, and maintain manual test plans and test cases for features across the full product
- Build and scale automation frameworks for regression, integration, and end-to-end testing
- Test AI-powered features including LLM outputs, agent behavior, and model responses for accuracy, safety, and consistency
- Collaborate closely with developers and product managers to shift QA left in the development lifecycle
- Define and track quality metrics, identify risk areas, and surface issues early before they reach production
- Develop evaluation strategies for non-deterministic systems — including prompt variation testing and output validation pipelines
- Participate in sprint planning and retrospectives as the quality voice on the team
What we're looking for
- 2+ years of professional experience in software quality assurance, covering both manual and automation testing
- Hands-on experience testing AI agents, LLM-powered features, or ML models — understanding their non-deterministic nature is a must
- Proficiency in at least one modern automation framework (Playwright, Cypress, Selenium, or equivalent)
- Strong API testing skills using tools like Postman, REST Assured, or similar
- Experience working in agile environments with CI/CD pipelines and integrated test gates
- A methodical, curious mindset — you find edge cases others miss, and you document everything clearly
- Solid communication skills to write clear bug reports, test documentation, and quality summaries
Bonus points if you…
- Have experience evaluating agentic workflows, multi-step reasoning chains, or RAG pipelines
- Have worked with LLM evaluation frameworks like RAGAS, LangSmith, or Promptfoo
- Use AI coding tools like Cursor or Claude in your daily QA workflow — we actively encourage it
- Have built custom eval harnesses or synthetic test data generators using LLMs
AI-first culture
We don't just build AI — we test it rigorously. Our QA engineers use Cursor and Claude to move faster, write better test cases, and automate the repetitive work. If you're already approaching quality assurance with AI tools in hand, you'll feel right at home.