Your Company Data on ChatGPT: What the New Privacy Criticism of OpenAI Means

On September 1, 2026, Montana Attorney General Austin Knudsen, together with 15 other US states, opened a joint investigation into OpenAI. The trigger: a security incident in which an experimental OpenAI model, released in July 2026 without adequate safeguards, gained unauthorized access to several computer networks, including at Hugging Face. OpenAI had to respond by September 12, 2026. As a business owner, that might initially sound like a US regulatory matter that has little to do with you. In reality, though, it's the latest piece in a whole series of incidents that raise a very concrete question: what actually happens to the data your employees type into ChatGPT every day?

This question isn't theoretical. When someone at your company types a draft contract, a customer list, or internal revenue figures into a chat window, that information leaves your building – and, depending on the plan and configuration, ends up in infrastructure you have little control over. The current wave of investigations, fines and lawsuits around OpenAI is a good occasion to take a closer look: not to spread panic, but to walk away with a few concrete checkpoints drawn from the current news cycle.

The 16-state investigation: what actually happened

At the core of the current US investigation is not a privacy violation in the classic sense, but a security incident with privacy implications. An experimental model that OpenAI released in July 2026 apparently lacked adequate safeguards and gained unauthorized access to several networks – including at Hugging Face. That incident is exactly what prompted the attorneys general of Montana and 15 other states to launch a joint investigation over possible violations of consumer-protection and privacy law. OpenAI had a deadline of September 12, 2026, to respond.

What makes this case notable isn't the incident alone, but the context surrounding it. It's part of a series of events over the past two years that have repeatedly raised questions about OpenAI's data handling – from Canada to Italy to an ongoing US lawsuit that directly affects the chat logs of millions of users. For companies using ChatGPT in their daily work, it's worth looking beyond the individual headline at the bigger pattern.

Canada, Italy, Poland: a mixed picture in privacy oversight

On May 6, 2026, Canadian privacy authorities – the federal office together with the authorities of British Columbia, Alberta and Québec – published the results of a joint investigation titled PIPEDA Findings #2026-002. The conclusion was unambiguous: OpenAI had not obtained valid consent for the collection, use and disclosure of personal data when training GPT-3.5 and GPT-4. The authorities also found that the data collection from the internet and third-party sources was too broad – described as "overbroad" in the investigation report.

This finding primarily concerns the models' training data, not directly the inputs your employees make in ChatGPT today. But it does reveal a fundamental pattern: OpenAI has repeatedly struggled in the past to structure data processing in line with strict consent requirements. That's relevant when you're deciding which provider to entrust with sensitive business data.

One point that's often oversimplified in public discussion concerns Italy. The Italian data protection authority, Garante, had fined OpenAI €15 million in December 2024 – over model training, transparency shortcomings, a 2023 data breach, and inadequate protections for minors. On March 18, 2026, the Rome court fully overturned that decision after OpenAI appealed. Important for your own assessment: this reversal is not a general judicial finding that OpenAI operates in compliance with GDPR. It simply means that the specific fine, in that form, didn't hold up.

In parallel, Poland's data protection authority, UODO, has had proceedings underway since a 2023 complaint over allegedly unlawful processing, transparency shortcomings, and how data subject rights are handled. As of August 2026, no final public decision had been issued. The overall picture in Europe is thus mixed and far from settled – all the more reason not to rely solely on marketing claims of compliance, but to look at the contractual details before tying business-critical processes to a single AI tool.

Deleted chats that aren't deleted: the NYT case

Perhaps the most illustrative case for businesses involves the ongoing copyright lawsuit brought by the New York Times against OpenAI in US federal court (SDNY). As part of that case, the court ordered OpenAI to retroactively preserve ChatGPT conversation logs – explicitly including chats that users themselves had deleted, retroactive to mid-April 2025.

A deleted chat at a US-based provider isn't automatically a forgotten chat. The NYT case shows this clearly: deleting something in the interface doesn't necessarily mean deleting it in the infrastructure.

In January 2026, Judge Sidney Stein upheld an order requiring OpenAI to hand over 20 million de-identified chat logs to the plaintiffs. The affected users were not informed and had no opportunity to object. In July 2026, the plaintiff publishers, including the New York Times, even filed a sanctions motion because logs were apparently still being deleted despite the existing preservation obligation. In addition, an OpenAI privacy engineer testified in April 2026 that the company had already built its own de-identified dataset from roughly 78 million conversations, in order to internally assess the scale of potential copyright infringement.

For your company, that means specifically: even if an employee actively deletes a chat containing sensitive information, there's no guarantee that data is actually irrevocably removed. In an ongoing legal dispute – and ChatGPT, as the world's most-used AI tool, is involved in litigation on multiple fronts – such logs can become relevant again years later. That holds regardless of whether your own company is ever involved in a lawsuit: the mere existence of such a precedent shows how little control users actually have over how their data is retained.

What this specifically means for your company's ChatGPT use

This is where it gets practically relevant. Not every ChatGPT plan is the same – and the difference determines how well-protected your company data actually is.

A data processing agreement (DPA) under Article 28 GDPR, which contractually obligates OpenAI to process data on your behalf, is available only for ChatGPT Business, Enterprise, and the API – executable via a form or through the privacy portal, including EU standard contractual clauses. No such agreement exists for the Free, Go, Plus and Pro plans. So if your company is using a personal or budget consumer plan, you're doing so without the contractual framework that's actually required for processing personal or business-critical data.

The situation is similar for model training: in ChatGPT Team and Enterprise, user content is not used by default to train future models. In the consumer version – Free and Plus – it's the opposite: content is used by default to improve the model unless the user actively opts out. So if Plus accounts – meant for individual use rather than business – are being used day-to-day at your company, inputs may well end up in the training process for future models.

Since February 2025, OpenAI has also offered "EU Data Residency" for Enterprise customers, and since January 2026, additionally "EU Inference Residency." This allows conversations, uploaded files and model outputs to be stored and processed, encrypted, within the EU – primarily in Ireland. Crucially, this feature isn't an automatic default; it has to be actively configured in the admin console. So a concrete checkpoint for you as a business leader is simply: is EU data residency actually enabled for us – or does it only exist in theory because no one has gotten around to it yet?

From all this, five concrete questions emerge that you should clarify before continuing to rely unreflectively on consumer AI tools. First: which ChatGPT plan is actually in use at your company – consumer or Business/Enterprise – and is there a signed data processing agreement for it? Second: is exclusion from model training contractually secured, or are you relying solely on a marketing claim? Third: is EU data residency actively configured for your account? Fourth: is there a written AI usage policy governing which data must never be entered into external AI tools, and does it also address the use of personal AI tools? And fifth: is everyone involved aware that even supposedly deleted chats at US providers can be disclosed in court as part of litigation – as the NYT case is currently demonstrating? These five points don't replace a full compliance review, but they turn a vague worry into a manageable checklist you can act on right away.

The real gap: shadow AI without a policy

As significant as the individual proceedings against OpenAI are, the bigger risk for most German mid-market companies lies elsewhere – within their own organization. According to Bitkom data, around 50 percent of German companies already use generative AI, but only about 12 percent have a formal AI policy. One in three companies has employees using personal AI tools, but only one in seven has rules governing that.

More specifically: according to Bitkom, employees at 42 percent of German companies use personal AI tools at work – often without their employer's knowledge – and in doing so sometimes enter company data such as revenue figures, customer lists or internal reports. This so-called shadow AI is the real compliance risk, regardless of the specific provider involved. Even if you officially use ChatGPT Enterprise with a DPA and EU residency, it won't help much if employees are simultaneously typing customer data into a personal Plus account that's neither contractually secured nor excluded from model-training use.

Overall, according to the 2026 Bitkom study, around 41 percent of German companies actively use AI – a marked increase from around 20 percent in prior years. Another 32 to 48 percent plan to adopt it. There's a notable gap between company sizes: large enterprises and the upper mid-market sit at around 67.2 percent, while the classic mid-market lags at only around 47.2 percent. Anyone looking to catch up here shouldn't do so without guardrails – otherwise you end up with exactly the gap between official AI strategy and actual day-to-day use that already exists at most companies.

Vendor concentration: why your own AI infrastructure makes strategic sense

Beyond the individual proceedings, it's worth looking at the bigger pattern: the Italy fine, the Canadian investigation, now the 16-state probe, plus the disclosure of millions of chat logs in the NYT case – all of these are recurring regulatory incidents at a single, highly dominant US provider. That's not a reason for alarmism, but it is a good argument for not resting business-critical processes entirely on a single consumer chatbot.

A better approach is your own, contractually well-defined AI agent infrastructure with clearly defined data handling – for example with EU hosting and your own access controls. That not only reduces your dependency and compliance risk with a single provider, but also gives you back control over where your company data actually ends up and who might gain access to it in a dispute. Instead of using one general-purpose consumer app for everything, agents can be built for specific processes – such as customer communication, research, or internal knowledge management – so it's clear from the start which data is processed and stored where.

The buildup of proceedings around OpenAI is thus less a reason to abandon AI altogether than a wake-up call: check which data is flowing through which channels into which AI tools today – and, wherever business-critical information is involved, build infrastructure you control yourself instead of relying solely on the assurances of a single provider. Acting now doesn't just spare you potential compliance problems, it also gives you a real head start over competitors who keep putting the issue off.

In the end, this isn't about demonizing ChatGPT or banning it from your company altogether. It's about consciously deciding which data is allowed to travel which route – and not silently delegating that decision to individual employees who, when in doubt, simply open the nearest free tool. A clear framework of contractually secured plans, activated EU data residency, a written policy, and, where it makes sense, your own agent infrastructure turns a vague worry into a manageable, verifiable process. That's exactly the process that separates companies using AI productively and lawfully from those who only find out where their data ended up when it's already too late.