Episode Details

The Federal Trade Commission ordered a California photo app called Everalbum to delete the photographs it had collected and every facial recognition model it had built from those photographs, and it issued that order in January 2021, two years before most founders had heard the phrase generative AI. The remedy is called algorithmic disgorgement, and it is the clearest indicator regulators have given that a company cannot keep the output of improperly used data even after deleting the input. Everalbum had told users that facial recognition required their consent while running the feature by default for most accounts, then combined the resulting face data with public photo sets to build recognition technology it later sold. The Commission’s order reached past the photographs into the algorithm itself. That distinction, between deleting data and deleting what a company built from the data, is the one every startup with a five year old vendor agreement needs to understand right now.

Here is the part that should concern a founder more than the Everalbum case itself. Everalbum got caught because it lied about consent. Most startups have never lied to anyone. They signed a standard software as a service agreement, the kind every vendor sends, containing a clause granting the vendor the right to use customer data “to provide, maintain, and improve the services.” That clause has existed in software contracts for two decades, written for a world of bug fixes and uptime monitoring. Attorney Juanita DeLoach called it out directly in a March 2026 analysis: the phrase “to improve our services” is broad enough to cover AI training without ever using the words artificial intelligence or machine learning. A vendor does not need to amend that agreement to start training a model on the data it already has permission to use. It only needs to decide that model training counts as improving the service, and the clause the startup’s general counsel approved in 2019 already says yes.

Two more clauses do the rest of the work, and both hide in plain sight. The feedback clause grants the vendor a perpetual, worldwide, royalty free license to use any suggestion or correction a customer provides, language written originally so a vendor could implement a customer’s product idea without a later ownership fight. Applied to an AI feature, that same clause captures every correction a user makes to a generated draft, every refined prompt, every annotated response. A team correcting a vendor’s AI generated contract draft, trying only to fix that one document, may be handing the vendor a license to train on the correction for every other customer who uses the tool afterward, competitors included. The third clause, permitting the vendor to use “aggregated or de-identified data,” looks the safest and gets the least scrutiny in a contract review, which is exactly why vendors lean on it hardest. The European Data Protection Board addressed the technical claim behind that comfort directly in Opinion 28/2024, published in December 2024: a trained model is not automatically anonymous just because a de-identification step happened somewhere in the pipeline. The Board requires proof that it is very unlikely anyone could extract personal data from the model or reconstruct it through targeted queries, assessed case by case. A contractual promise to de-identify data before training does not, on its own, meet that bar, and the promise still functions as a license whether or not the underlying anonymization actually holds.

Zoom gave the market a live demonstration of what happens when this goes wrong in public. In March 2023 the company quietly added terms permitting it to train its own and third party AI models on customer audio, video, and chat content. Nobody noticed until August 2023, when the terms surfaced and set off a rapid revolt among Zoom’s own enterprise customers, the kind of company that reads its own vendor contracts closely. Zoom walked the