“Clarifying METR’s Auditing Role [linkpost]” by ChanaMessinger

This is a link post. METR clarifying what auditing it has or hasn't done, and what its plans are for the future, seems very important for understanding the landscape. Excerpt: METR has not intended to claim to have audited anything, or to be providing meaningful oversight or accountability, but there has been some confusion about whether METR is an auditor or planning to be one. To clarify this point: METR's top priority is to develop the science of evaluations, and we don’t need to be auditors in order to succeed at this. We aim to build evaluation protocols that can be used by evaluators/auditors regardless of whether that is the government, an internal lab team, another third party, or a team at METR.    We should not be considered to have ‘audited’ GPT-4 or Claude. Those were informal pilots of what an audit might [...] --- First published: June 4th, 2024 Source: https://forum.effectivealtruism.org/posts/86LEneMmF4Dk6pYfc/clarifying-metr-s-auditing-role-linkpost --- Narrated by TYPE III AUDIO.