Notion AI chats exposed in breach at AI provider Fireworks

Notion has notified customers whose Notion AI chats were exposed in a breach at one of its vendors, Fireworks AI. Notion used the Fireworks platform to serve open-weight models and to run classification jobs for Insights, a feature that shows customers how AI is used in their own workspace. According to Notion's security advisory, the incident affects less than 1% of workspaces.

What the exposed chats contain

The exposed data consists of Notion AI chat transcripts that Notion had sent to Fireworks for Insights. For the vast majority of affected workspaces, Notion says the transcripts were "unattributable," meaning they were not labeled with who wrote them or which workspace they came from. They also did not include tool call results. The transcripts still contain what users typed, though. On November 11, Notion revoked Notion API keys that it found in affected messages during its investigation.

Fireworks first alerted Notion on November 2 to a dataset that appeared to contain only synthetic test data. By the morning of November 3, Notion had disconnected its systems from Fireworks and deleted all of its data there. On November 6, Fireworks reported that customer data may have been accessed, and on November 8 it confirmed the final dataset, which included real customer chats. Notion began contacting affected customers on November 9.

How the attackers got in

Fireworks described the sequence in an update published on November 9. A third-party cloud provider that hosted Fireworks research and development machines had a security incident on October 23. The next day, the attacker used that access to reach Fireworks development machines and ran an unauthorized workload there. The stolen data includes resource metadata, billing information and, for some customers, parts of datasets and incomplete model weights. Fireworks says it does not store payment card numbers. An earlier intrusion in early October had exposed stored secrets such as API keys.

Fireworks links the attack to a financially motivated group that sells unauthorized access to large language models, a practice known as LLMjacking. Google Threat Intelligence Group and Mandiant had warned about its rise on November 2, and we recently reported on stolen AI access being sold on the black market. Fireworks brought in Palo Alto Networks Unit 42 to help with the investigation.

What Notion users should do now

If neither you nor your workspace owner has heard from Notion, the company says no action is needed for now. Owners of affected workspaces can review the data under Security incident review in Notion's web interface and send members a link to download their own transcripts, BleepingComputer reports. Notion recommends treating this data as if it were exposed. Any passwords, API keys or other credentials pasted into Notion AI during the incident window should be rotated.

Insights stays disabled while the investigation continues, and Notion no longer serves open-weight models through Fireworks. The case shows that a Notion AI chat can end up with a subprocessor and, from there, with criminals. If you use Notion with AI or are weighing alternatives, our comparison of Obsidian and Notion looks at which tools let AI in at all.