AI Testing Requirements: How to Spot Trustworthy Providers

Since June 2, 2026, the US has had an executive order that reshapes the relationship between government and AI providers. And as of this week, the White House is apparently negotiating with OpenAI, Google, and Anthropic over the concrete implementation: a framework for pre-release testing of new AI models before they're published. An official announcement was expected for the week starting July 7, 2026 - as of the time of this article, it had not yet materialized. But one thing is already clear: for you as an executive or decision-maker buying AI tools and agents, this creates a new, publicly visible benchmark for vendor selection. We'll show you what's behind the headline, what it doesn't mean, and the checklist you can use to spot trustworthy providers today.

Not a testing mandate, but a voluntary framework with incentives

The term "testing requirements" used in this article's title is deliberately blunt - and that's exactly why we should put it in context right away. The current development is grounded in the executive order "Promoting Advanced Artificial Intelligence Innovation and Security," which President Trump signed on June 2, 2026. It involves the Department of Commerce with its Center for AI Standards and Innovation (CAISI), the Department of Defense, the Department of Homeland Security with CISA, and the Office of Management and Budget.

Central to it is the newly introduced concept of a "covered frontier model": providers can grant the government access to a particularly capable model for up to 30 days before its release, so agencies can assess possible national security risks - especially in the areas of cyberattacks and biological and chemical weapons. The order explicitly states that this is not meant to create mandatory licensing, prior approval, or a formal certification requirement.

Participation remains formally voluntary - but is, in practice, tied to tangible incentives, such as access to federal cloud contracts and government AI procurement. Those who don't play along lose access to one of the largest buyers in the world.

It's exactly this coupling that turns a voluntary program into something the big providers can hardly afford to ignore in practice. The responsible testing authority is CAISI, the successor to the former "AI Safety Institute" within NIST. CAISI has already signed testing agreements with OpenAI, Anthropic, Google DeepMind, Microsoft, and xAI. According to press reports, over 40 model evaluations have already been carried out, some classified through the interagency "TRAINS Taskforce," focused on cyber, bio, and chemical weapons risks. We could not find an official primary source confirming this figure during our research - we're therefore explicitly presenting it as press coverage, not a confirmed fact.

Also still open is the question of exactly what compute level or benchmark scores would qualify a model as a "covered frontier model" in the first place. The executive order sets a 60-day deadline for this, which lands around August 1, 2026. According to reports, companies are pushing for high thresholds so that as few of their models as possible fall under the rule; government agencies are pushing for low ones, so that as many models as possible get tested. As of July 8, 2026, how that dispute will play out is not yet decided.

The key thing for you here is this: the framework directly affects only a handful of providers at the very top of model development - companies whose models are large enough to count as "frontier." Most of the AI tools actually in use at your company, however, are built on exactly these frontier models, for example via API connections to GPT, Claude, or Gemini models. The rigor with which these base models are tested therefore has an indirect effect on every product, every agent, and every automation built on top of them - even if the software vendor you actually sign a contract with doesn't train any frontier models of its own.

Why this already matters for your vendor selection today

As a mid-sized company, you generally don't buy frontier models directly - you use them through products, APIs, and agents built on top of them. Still, this development affects you directly, for three reasons.

First, it shifts the benchmark for what counts as a "thoroughly tested" provider. Until now, you had to rely almost entirely on a provider's self-reporting when asking whether an AI model had been adequately tested. With CAISI, an external - if partly classified - review body now exists for the first time, one you can reference in conversations with your providers: "Has your model been tested by CAISI or a comparable process?" is, from now on, a legitimate, concrete question in your due diligence.

Second, the current case exemplifies how quickly a provider's status can change. Reports indicate that OpenAI's GPT-5.6 rollout was delayed and restricted at the US government's request over safety concerns - currently, according to reports, only around 20 government-vetted partner organizations have access to the GPT-5.6 variants. Anthropic, too, lost a $200 million Pentagon contract in 2026 after refusing a contract clause on autonomous weapons use without human control, and was temporarily even classified as a "Supply Chain Risk to National Security." Its current participation in the new voluntary testing standards is seen as a sign of rapprochement between Anthropic and the US government. For you as a buyer, that means: a US provider's regulatory status is not a fixed quantity - it can shift within a few months.

Third, the US framework explicitly does not replace your obligations under the EU AI Act. Both levels run in parallel, with different aims - more on that shortly.

There's a fourth, more strategic reason on top of that: if you start systematically asking providers about safety and testing criteria today, you build yourself an internal documentation framework that you'll need anyway - for internal approval processes, for customer inquiries about your own AI use, and for communication with your works council or legal department. Whoever waits to ask these questions until a regulator or a customer asks them first is under time pressure. Whoever builds them into regular vendor selection now already has an answer ready when it matters.

The checklist: six questions to ask your AI provider

From the existing voluntary safety frameworks of the major providers - Anthropic's Responsible Scaling Policy with its AI Safety Levels ASL-1 through ASL-4+, OpenAI's Preparedness Framework with risk levels from low to critical, and Google DeepMind's Frontier Safety Framework - as well as established reference standards such as the NIST AI Risk Management Framework and the certifiable ISO/IEC 42001, a practical checklist can be derived. Ask your provider these six questions before signing a contract:

• Is there a public safety or risk framework? Providers who disclose their risk levels and escalation thresholds show that they think about safety proactively, not only on request.
• Has independent red-teaming been conducted? External attack and misuse testing is a strong indicator of serious scrutiny, not just internal self-affirmation.
• Is there a model or system card with benchmark results? Transparent performance and risk data can be compared against competitors.
• Is the provider a signatory of the EU GPAI Code of Practice? Around 24 to 26 providers have signed so far, including Anthropic, OpenAI, Google, Microsoft, Amazon, IBM, Mistral AI, Cohere, and Aleph Alpha. Meta has not signed; xAI signed only the safety and security chapters.
• Does the provider align with the NIST AI RMF or ISO/IEC 42001? While the NIST AI RMF remains voluntary with no official certification, ISO/IEC 42001 is a certifiable management system - ask specifically about a certificate, not just an "alignment."
• Are there transparency reports and clear reporting channels for security incidents? California's SB 53 provides a workable reference framework here, with 15-day or 24-hour deadlines for acute threats, that providers outside California should be measured against too.

Important for applying this checklist in practice: you won't get a complete answer to many of these questions from smaller or specialized providers - that's not automatically a disqualifier, but a conversation starter. What matters is whether a provider can and will give any answer at all, or visibly dodges the topic.

A practical tip from our consulting work: build the six questions into your RFPs or vendor comparisons as a fixed block, rather than only asking them verbally in an initial conversation. Written answers are easier to document, compare, and - if necessary - present to regulators or customers later. Especially for points four and five, the GPAI signature and the alignment with NIST AI RMF or ISO/IEC 42001, it's also worth checking the date: frameworks and certificates go stale, and a provider that committed to a standard two years ago doesn't necessarily still maintain it actively today.

Two parallel worlds: don't confuse the US framework with the EU AI Act

A common mistake in practice: when people hear that a US provider "is now being tested by the government," they quickly assume that provider is automatically EU-compliant. That's a fallacy you should avoid in your own compliance review.

The new US framework is, as of today, voluntary, focused on national security, and aimed primarily at cyber, bio, and chemical weapons risks in frontier models. In parallel, California's Transparency in Frontier Artificial Intelligence Act (SB 53) has been in effect since January 1, 2026: major developers of frontier-capable models with training compute above 10^26 FLOPs must publish an annual safety framework, deliver transparency reports before model releases, and report critical safety incidents to the California Office of Emergency Services within 15 days - or 24 hours in cases of acute danger. Violations can carry fines of up to $1 million per violation.

At the EU level, meanwhile, there's the General-Purpose AI Code of Practice under the AI Act, which the aforementioned 24 to 26 providers have voluntarily signed on to. Actual enforcement by the European Commission - with requests for information, model access, and, in extreme cases, recalls - only begins as of August 2, 2026. That means: even if a provider ticks all six boxes on the checklist above, your company in Germany remains solely responsible for your own EU AI Act compliance - for example, when it comes to classifying the risk of your own AI applications. The US testing standards can be an additional trust signal, but they don't replace your own legal review.

In practice, that means checking both levels separately. A provider's US testing participation answers the question "How carefully was this model tested for safety risks?" Your own EU AI Act review answers the question "What does my company in Germany need to do if I deploy this model?" Both questions belong in your vendor documentation, but they're not interchangeable.

Another difference lies in who the rules are addressed to. SB 53 and the EU AI Act target the model developers themselves directly, obligating them to disclosure and reporting channels. The new US federal framework, by contrast, primarily addresses the relationship between provider and government - it creates no direct obligations toward you as an end customer. That means: even if a model has successfully passed CAISI review, you as a company don't automatically gain any right to inspect the test results. It becomes all the more important not to rely on your providers' claim that "we're being reviewed by the government anyway," but to demand concrete, contractually assurable statements instead.

What's still unclear - and why that's no reason to wait

Honestly: a fair amount of uncertainty remains. The actual announcement of the new US testing standards had, according to available sources, not yet been finalized as of this research on July 8, 2026. Reports spoke of "advanced negotiations" and an announcement expected for the week starting July 7. Concrete benchmark names and exact thresholds for "covered frontier models" were nowhere found to have been published as of our research - the benchmarking process is, per the executive order, even partly intended to be classified, which is likely to limit the transparency end customers would want.

So should you wait before revising your vendor selection? No. The six checklist questions in this article deliberately don't rely on the still-unknown US thresholds, but on already-existing, verifiable criteria: public frameworks, red-teaming, model cards, GPAI signature, management-system standards, and incident reporting channels. These criteria hold regardless of how the US negotiations play out in the coming weeks. What may change are the details - for example, whether CAISI test reports become visible to customers in some form, or exactly where the compute threshold for "covered frontier models" ends up. Treat the checklist, then, as a living document: solid as of today, but likely to be supplemented with more concrete US criteria in the coming months.

Conclusion: trust is becoming measurable - use it

The "testing requirements" in the headline are, strictly speaking, a voluntary framework with strong economic incentives. But regardless of how the details shake out in the coming weeks, the direction is clear: the trustworthiness of AI providers is increasingly being tied to verifiable criteria rather than marketing promises. For you as a mid-sized company, that's an opportunity: instead of relying on gut feeling or the size of a logo, you can now ask targeted questions - about frameworks, about red-teaming, about certificates, about reporting channels. Providers who answer these questions openly deserve your trust more than those who dodge them. And that distinction will only become more important as regulatory attention grows in the US, in California, and in the EU.