Google's Pixel AI Image Creation and Editing Tools Are Kind of Terrifying

Google last week introduced the Pixel 9 smartphone lineup, with a selection of new devices that are focused on AI. All of the Pixel 9 phones support Gemini AI, and Google has also added AI-based image generation and editing tools. Reviewers have been testing the new smartphone features, and the AI image generation is probably going to turn into a PR nightmare for Google based on what's come out so far.

Google introduced an AI image generation app for the Pixel phones, called Pixel Studio. It's designed to create stickers and images using text-based prompts, and it looks and sounds a lot like Apple's planned Image Playground. So far, reviewers have been able to create all manner of questionable images with Pixel Studio, ranging from a Nazi version of Spongebob to Elmo with an AK47.

Google said that there were "safety checks" in place to keep Pixel Studio from being "used nefariously," and there are some. Pixel Studio will not create images of humans, and Google is tweaking the AI in response to reviews. After speaking with Google, Digital Trends was no longer able to get Pixel Studio to create cartoon characters using cocaine or dressed as German soldiers.

Even more concerning than Pixel Studio is Google's "Reimagine" tool, which can add objects to photos that you've already taken. The Verge used it to add corpses, bombs, drugs, and disasters to images, and objects included in photos look so realistic that it's difficult to tell when an image has been edited. Google is able to do a pretty good job matching lighting and perspective to the original photo, and there are no watermarks or flags on social media. Google does add a metadata tag, but that's easy enough to eliminate with a screenshot.

You can, of course, use Reimagine for adding sunsets and rainbows to your image, just like you can use Pixel Studio for creating fun pictures, and both AI features work great, so these tools aren't all bad. Android Authority shared a post with Oreo pizza, a giraffe surfing, and kittens playing basketball, for example. As The Verge points out, you could always add a body to an image or create a picture of Elmo with a gun, but it would take some Photoshop skill and time to do so. With the Pixel phones, it takes just a thought and a few seconds to create that kind of image, and it's right there on the smartphone for anyone to use.

In statements to The Verge and Digital Trends, Google said that it designed its generative AI tools to "respect the intent of user prompts" which can lead to the creation of content "that may offend" when the user asks for it. But Google claims that there are "Terms of Service" on what content is not allowed, and that safeguards will be continually refined.

Apple hasn't yet introduced its image generation tools, ‌Image Playground‌ or Genmoji, and the media surrounding the Pixel 9 launch gives us some insight into what the response might be like if Apple doesn't get it just right. With AI, it's likely that people are going to be able to find workarounds even if Apple puts numerous safeguards in place. Apple so far has shied away from photorealistic AI images, but Google is going all in, and it's going to be much harder to trust photos on social media and the internet going forward.