Google's SynthID watermark is hard to break, but it doesn't solve AI disinformation
Ars Technica · LC · trust 31/100
Real problems Google’s SynthID watermark is hard to break, but it doesn’t solve AI disinformation Deciding what’s real on the Internet won’t be easy in the future.
51 Credit: Aurich Lawson Credit: Aurich Lawson Text settings Story text Size Small Standard Large Width * Standard Wide Links Standard Orange * Subscribers only Learn more Minimize to nav The scale of AI-generated media can be hard to grasp.
Starling Lab, a research collaboration from Stanford University and the University of Southern California, estimates that it took until 1975—149 years after the invention of the camera—for humanity to create 1.5 billion images. It took generative AI just 18 months to do the same. And AI didn’t stop there.
This spring, Google announced at I/O that its tools had been used to create more than 100 billion AI images and videos in just a couple of years, and Google is far from the only source of AI content. The company coupled this staggering statistic with a slew of partnerships to expand the use of its SynthID watermarking technology, which can be used to label AI-generated content and potentially help people identify what’s real.
The company says SynthID is robust enough to survive edits, and we’re about to see that tested as OpenAI, Runway, Nvidia, and others begin using it. Can invisible watermarks solve the AI content conundrum?
There are currently two approaches to labeling AI content: invisible watermarks like SynthID and metadata schemas like the Coalition for Content Provenance and Authenticity (C2PA). Google uses both to label its AI content, but C2PA is not intended to hide in plain sight like a watermark. It’s cryptographically secure, so you can’t fake it, but C2PA is trivially easy to strip out. Simply editing and saving an image or taking a screenshot can remove C2PA.
Watermarks like SynthID are encoded in the pixels of an image or video or in the waveform of an audio clip. Content that gets passed around the Internet degrades from compression, resizing, and edits, but Google says SynthID should still be present even in well-worn memes. Google DeepMind scientist Pushmeet Kohli explained to Ars that the team put a great deal of work into ensuring SynthID was a durable way to label AI content.
“Through the whole development process, we sort of assumed that a technology like this will be attacked,” said Kohli. “So we did a lot of research in making SynthID robust to different kinds of transformations. Whether people are adding some sort of filter or cropping the image, we used these transformations and made sure that the detector was robust against them.”
Google has been reluctant to provide much technical detail on SynthID’s functionality beyond the original paper, so it’s worth testing these claims as the technology expands across the AI landscape. But who has the time to wait for AI images to degrade naturally as they get passed around the Internet?
I used the Python Pillow library to simulate data loss from repeated sharing and downloading at a vastly accelerated rate. The Python script chooses random compression and resizing values within set ranges to apply to the test image, then uses the output image as the basis for the next iteration.
This test focused on two types of AI images: one created entirely from scratch by the model and an original photo edited by AI. Both images generated by Nano Banana Pro include the SynthID watermark, and no amount of metadata editing can hide it.
A completely AI-generated test image. Google AI A test image edited with Google AI. Google AI A test image edited with Google AI. Google AI A completely AI-generated test image. Google AI A test image edited with Google AI. Google AI I started with the images above for the original and edited tests. We can probably all agree that neither one is real—politics is not metal, and my dog is not part dragon (take my word for it). I ran these images through the image-crushing machine hundreds of times, turning the crisp originals into barely recognizable blobs. Every 50 generations, I also created cropped versions of the images (without reencoding), removing potential SynthID pixels to further weaken the detection.
After 300 generations of simulated sharing, we’re left with these full-frame images:
A fully AI-generated image after 300 compression cycles (SynthID intact). Google AI An AI-edited image after 300 compression cycles (SynthID intact). Google AI An AI-edited image after 300 compression cycles (SynthID intact). Google AI A fully AI-generated image after 300 compression cycles (SynthID intact). Google AI An AI-edited image after 300 compression cycles (SynthID intact). Google AI And look at that—the SynthID watermark still works on both of them. You can check for yourself by uploading the images to Gemini and asking for a SynthID check. You can even take a screenshot of the full image, and the system will still label it as SynthID because the special pixels transfer over to the new file.
Because SynthID pixels are spread throughout the image, they’re also resistant to cropping… at least to a point. This is where we finally find the limits of SynthID. After 300 compression generations, removing a few pixels from the border of our test images finally broke SynthID (both edited and fully AI). These versions (see below) have been cropped by 20 percent, but that’s enough to render SynthID undetectable. A larger 50 percent crop can break SynthID a bit earlier, at around 250 image compression iterations.
A small crop of this compressed AI image successfully removes SynthID. Google AI After a small crop, this heavily compressed AI edit no longer has SynthID. Google AI After a small crop, this heavily compressed AI edit no longer has SynthID. Google AI A small crop of this compressed AI image successfully removes SynthID. Google AI After a small crop, this heavily compressed AI edit no longer has SynthID. Google AI These images may be ugly, but there’s nothing (other than the content) to…
Read the original at Ars Technica →
Open in TruthVane →