Senior Software Engineer — Customer Files Orchestration

Job description
Senior Software Engineer — Customer Files Orchestration

CoCounsel Legal is seeking a Senior Software Engineer to help build and operate the Customer Files Orchestration Service – the system that makes a customer's own documents usable by CoCounsel Legal's agents.

Every file a customer brings to CoCounsel passes through this team: upload, storage, extraction, embedding, and indexing, orchestrated end-to-end. You make millions of customer files available to CoCounsel. Bringing a customer's content together with Thomson Reuters' world-class proprietary content and the latest frontier models – including TR's own custom large language models – is what makes our products highly differentiated and our agent experiences genuinely grounded. This team is a critical part of what makes that combination possible.

It is also, unavoidably, a hard distributed systems problem. Supporting millions of files means real engineering: dedicated workers, fair queuing so one busy customer cannot starve another, durable state, idempotent processing, and honest reasoning about consistency and delivery guarantees across systems this team does not own.

The team owns what it builds, designing, implementing, testing, deploying, scaling, and operating the service end-to-end. As a Senior Software Engineer you own a meaningful part of that service outright. You are eager to ship something you’re proud of and become someone the team trusts.

Key responsibilities:

Design & Technical Ownership

• Own the orchestration service end-to-end – Take an ambiguous problem, shape it into a design, write the ADR, build it, ship it, and operate it. You and the team are accountable for the outcome, not just the pull request.

• Make and defend the calls that define this domain – Where eventual consistency is safe and where only a strongly-consistent read is correct; at-least-once with idempotent processing versus the real cost of exactly-once; lease and heartbeat models for crash recovery; compare-and-swap state transitions across asynchronous service boundaries.

• Build the queuing and flow-control machinery – Fair queuing so one busy tenant cannot starve another, rate limiting, backpressure, circuit breaking, dead-letter handling, and autoscaling.

• Treat our APIs as a product – CoCounsel's file and workspace APIs serve a growing set of consumers beyond orchestration. Versioned contracts, documented behavior, backward compatibility, and deliberate change management are part of the work, not paperwork after it.

• Design for correctness at scale, not correctness in a test – Anticipate the failure modes that only appear under concurrency and load and build systems where whole classes of bugs cannot occur rather than fixing them one at a time.

Reliability & Operations

• Operate what you build – Take a primary on-call rotation for the service, lead incident response, and drive the post-mortem that keeps the same failure from recurring.

• Make the system explain itself – Instrument and trace well enough to tell "we never notified downstream" apart from "we notified downstream too early", and to answer a customer's "where did my file go?" in minutes rather than days.

• Reduce operational toil deliberately – Automate the recurring manual fix and treat a runbook step that exists only in someone's head as a defect.

Technical Leadership & Collaboration

• Raise the bar around you – Mentor engineers through code review and design review. Bring positive energy to the team.

• Work well across boundaries – Some stages of this pipeline are owned by other teams. Partner with the storage, extraction, indexing, connectors, and front-end teams to ensure we all ship something great.

• Fully integrate Architecture and DevOps – You are part of a team where engineers do the system design. There are no DevOps engineers because everyone is responsible for developing and operating.

• Communicate trade-offs plainly – Make the cost of correctness, the meaning of a throughput number, and the risk in a deadline legible to product partners and to your own team.

Required Qualifications:

• Bachelor's or master's degree – Computer Science, Engineering, or a related field, or equivalent practical experience.

• 5+ years building production software – With a track record of owning significant components in systems that real customers depend on.

• Extensive AWS and production Python experience – This is a Python service on AWS, and you should be able to move confidently through both the code and the infrastructure.

• Concurrency in Python, and a healthy respect for how treacherous it is – You understand the GIL and what it does and does not protect you from, you have written and debugged production asyncio code, and you have been bitten by the classics: fire-and-forget tasks that swallow their own failures, blocking work smuggled onto the event loop, semaphores and pools that quietly cap throughput far below the worker count, and background tasks racing the state they are meant to be updating.

• Kubernetes in production – Our workers and APIs run as pods with event-driven autoscaling (KEDA). You have tuned scalers and resource requests, and you have debugged the real failure modes.

• Large-scale orchestration or data pipeline experience – Systems that move large volumes of work through multiple stages across service boundaries, where retries, partial failure, and long-running steps are the normal case rather than the exception.

• Consistency models – You can explain where eventual consistency is safe and where it silently corrupts a user-visible outcome, and you have debugged the difference in production.

• Delivery guarantees – At-least-once versus exactly-once, why idempotency is usually the real answer, and what it costs to get it wrong.

• Queues, fair queuing, rate limiting, and backpressure – Practical, hands-on understanding, including multi-tenant fairness and noisy-neighbor isolation.

• API design instincts – Versioned contracts, clear ownership, backward compatibility, and treating an internal API as something other teams build on.

• Real operational ownership – You have carried the pager for something you built, debugged it under pressure, and come out with a fix rather than a workaround.

• Strong engineering fundamentals – Testing, code review, observability, CI/CD, and infrastructure-as-code as habits rather than obligations.

• Clear communication – You can write a design down, defend it, change your mind when the evidence says so, and explain the result to someone who does not share your context.
  

Preferred Qualifications:

• Documents and document management systems – Extraction, OCR, format handling, very large files, and the integration surface of third-party DMS platforms.

• Event-driven architectures and streaming – Kafka/MSK or similar, managed queueing such as SQS, and event-driven autoscaling.

• Durable workflow engines – Temporal, Step Functions, or similar, with a clear view of when they are worth adopting.

• Retrieval, embedding, or indexing pipelines – Supporting LLM or agentic applications.

• Prior exposure to the legal industry – A real bonus, and a meaningful head start on understanding what our customers actually need from their documents.

• Team topologies – This organization uses its vocabulary natively (stream-aligned teams, platform teams, and the boundaries between them) to reason about how services and teams are shaped.