What METR published on 20 October
METR reviewed the misalignment section of Anthropic’s August 2026 Risk Report. It broadly agreed with a low catastrophic-risk assessment for the covered models, primarily because of capability limits. It questioned whether the report’s methods would reliably identify higher risk. This concerns a defined set of models and catastrophic outcomes; it does not measure routine business error rates.[1]
METR describes this as a relatively lightweight review based on the internal report. The engagement began before Anthropic’s newer embedded-evaluator announcement and did not have that level of access. Anthropic retained publication-veto rights for this review; METR says no redactions or changes to its conclusions resulted from publication review.[1]
A newer access arrangement is a separate development
On 15 October, Anthropic announced that six METR employees had received company hardware, office space and internal access broadly comparable to its risk-assessment teams. It said METR could publish without a company veto, subject to predefined redaction categories whose application Anthropic determines. The announcement was clarified on 16 October. Those terms should not be retroactively attributed to the earlier review.[2]
Anthropic’s September announcement of work with Accenture also acknowledged that embedded evaluation was developing without settled access, reporting or funding standards. It said Anthropic would directly fund Accenture’s work. These differing arrangements matter when judging what independence means in a particular engagement.[3]
Kooya analysis: request an evidence boundary
For a Philippine business buying an agent, the practical document is a short evidence boundary attached to the proposal. It should identify the model version tested, the software wrapper, available tools, accessible data and the action types examined. A general assurance about a model provider cannot answer whether your integration can change a bank-detail field or expose a customer record.
Require the supplier to separate observed results from assumptions. An observation might be that a restricted test account could not submit a payment. An assumption might be that production roles will match that account. These have different owners: engineers can demonstrate the first, while operations must verify the second after deployment. Recording both makes a review useful rather than ceremonial.
This recommendation is Kooya’s implementation analysis. We have not inspected Anthropic’s internal systems, independently reproduced METR’s findings or tested a customer deployment for this article. The published review supplies a reason to ask better questions, not evidence that a specific Philippine system is unsafe.
Use our AI pilot evaluation checklist to develop the next set of practical questions.
Make assurance change when the system changes
A practical acceptance record should expire when material conditions change. Adding an email-sending tool, widening a document collection or switching the underlying model can invalidate a previous test even when the chat interface looks identical. Ask the supplier to name which changes trigger a repeat review and who may approve the new configuration.
Consider a hypothetical procurement assistant that originally compares public supplier catalogues. Giving it access to internal bid evaluations creates a different confidentiality boundary. Letting it send award notices creates a further authority boundary. Treat each extension as a new decision with its own test evidence instead of allowing the original pilot sign-off to cover every later permission.
A small team can keep this manageable with one page per workflow: permitted actions, prohibited actions, test date, failed cases, unresolved assumptions and owner. Link that page to the actual configuration. The goal is an accountable record that can be checked during an incident, not a long document that becomes detached from the software.
What to watch next
Kooya would look for future published assessments that clearly distinguish new access from old engagements and explain how conclusions respond to operational evidence. Procurement teams should also request failure descriptions, not just favourable summaries. A reviewer’s inability to inspect a component belongs beside the conclusion where decision-makers can see it.
The business decision remains proportional: a drafting assistant and an autonomous operator do not need identical acceptance criteria. Start by defining the loss an incorrect action could cause, then require evidence relevant to that loss. The presence of a respected evaluator is useful context; it cannot replace ownership of the workflow you deploy.