A testing supplier can become excellent at the contract and surprisingly weak at the product. Reports arrive on time. SLAs are quoted accurately. Every role is mapped to a responsibility. Then somebody asks how a critical customer journey actually works and the room develops an interest in the ceiling.
Product knowledge matters because useful testing depends on understanding where failure hurts, how data moves, which workarounds are real and which “edge cases” happen to customers every day. A supplier that can quote clause 7.4.2 from memory but cannot explain the checkout journey has achieved a very specific kind of expertise.
If contractual compliance looks stronger than delivery understanding, we can review the service and where the knowledge is actually sitting before the next release teaches everyone the expensive way.
Work out why the knowledge disappeared
High turnover, constant rotation, offshore hand-offs, role fragmentation and aggressive utilisation can all erode product knowledge. So can the client treating the supplier as disposable execution capacity and withholding access to the people who understand the business.
Do not diagnose laziness before looking at the service design. A tester moved between four products every fortnight cannot build deep context by force of personality.
Test understanding with real risk
Ask the supplier to explain the most important customer journeys, data flows, failure modes, known operational pain and areas where previous releases have gone wrong. Then compare that with what the test approach emphasises. If those are missing, you can end up with a supplier meeting SLAs without delivering a useful service.
You are not running a pub quiz. You are checking whether the service knows enough about the product to make intelligent choices.
Find where the experienced people went
Many services have a small number of people who genuinely understand the system while the majority execute around them. Make those dependencies visible. If two individuals leaving would collapse effective testing, the service has a resilience problem even if the headcount looks healthy.
Contract compliance is not product knowledge, however beautifully formatted the compliance tracker may be.
Senior names in a proposal are not the same as senior capability in daily delivery. Check who is actually available when difficult decisions need to be made.
Make knowledge retention part of the service
Use shared product maps, risk histories, automation ownership, useful test charters, incident learning and deliberate handovers. Keep them practical. Nobody needs a 400-page knowledge repository that proves the knowledge was technically stored somewhere.
The organisation should also retain enough understanding to challenge the supplier. Outsourcing execution does not outsource responsibility for knowing your own service.
Challenge contract theatre
A supplier may respond to questions by showing that every contractual obligation was met. That can be true and still miss the point. If the required service does not create sufficient product understanding, the contract itself may need to change.
Do not let discussions end at “not in scope” when the missing capability is necessary for safe delivery. Decide whether to add it, move it elsewhere or redesign the service.
Judge whether the relationship can learn
The important test is what happens after the gap is exposed. Does the supplier improve continuity, change staffing, build knowledge and challenge its own process? Or does it explain more elegantly why none of this breaches the agreement? You are paying for a testing service, not an escape room where the answer is hidden in the statement of work.
A supplier that learns can recover. A supplier that treats the statement of work as a shield may remain contractually correct all the way into the next incident.