AI vendor reviews need technical evidence, not better questionnaires

AI has made third-party risk more technical. The old questionnaire still matters, but it cannot be the whole review. If a vendor is processing sensitive data, generating security decisions, or exposing agents with tool access, the assessment needs evidence.

The useful questions are specific:

Data use: Is customer data used for model training, evaluation, or human review?
Access control: Who can see prompts, outputs, inference logs, and support tickets?
Retention: How long are prompts, outputs, embeddings, and logs stored?
Model change control: How are model updates tested, approved, and communicated?
Agent permissions: Can the product read or write to connected systems, and under whose identity?
Incident evidence: Can the vendor show detection, notification, and customer-impact workflows?

In my experience with 500+ risk assessments, the gap is usually not that a vendor refuses to answer. The gap is that the answer is policy-only. The vendor says encryption exists, but the evidence is a marketing page. The vendor says humans review outputs, but there is no auditable workflow. The vendor says data is isolated, but there is no architecture diagram or control owner.

For AI vendors, I would rather see a smaller questionnaire paired with stronger artifacts:

Data-flow diagram
Subprocessor list
Access review evidence
Encryption and key-management summary
Logging and retention policy
Model release/change process
Incident response runbook
Sample customer notification workflow

The useful upgrade for TPRM is not longer forms. It is better evidence, tighter scoping, and continuous review when the vendor becomes part of your control surface.