Over the last few months, many have proposed that the AI spam taking over Facebook is a great example of the “Dead Internet Theory,” which posits that large portions of the internet are made up of bots talking to bots, filtered through the lens of recommendation and engagement algorithms. Facebook is undeniably cooked, a decaying, depressing hall of horrors full of viral AI-generated content that seemingly gets worse every day.
But I do not think Facebook is the dead internet. Instead, I think it is something worse. Facebook is the zombie internet, where a mix of bots, humans, and accounts that were once humans but aren’t anymore mix together to form a disastrous website where there is little social connection at all.
I have spent more time than anyone I know endlessly scrolling through AI spam on Facebook. I have watched the evolution of Facebook’s AI spam go from slightly uncanny modifications of real images to the completely bizarre and obviously fake. I have done this from my own Facebook account, which I have had since 2005, as well as from two burner accounts I created specifically to track how AI-generated content is recommended on the platform and to see whether Facebook would put AI-generated images into my feed organically. I now use Facebook exclusively to see what kinds of bizarre AI content is going viral, and to attempt to figure out who is making it, why they are making it, and who is interacting with it.
Over the last few months, I have messaged roughly 300 people who have commented on AI-generated posts to try to ascertain if they are real, which has been an arduous task because Facebook keeps rate-limiting me. I have studied the profiles and activity of hundreds of Facebook users who are commenting on these images. I have also asked my friends and family, as well as 404 Media readers, to send me examples of AI-generated images they’ve seen on Facebook. I have spoken to Stanford University researchers who attempted to systematically study this phenomenon, talked to Facebook users who have been closely tracking the spread of AI since last year, and I have scanned through tens of thousands of viral AI-generated images on the platform. Definitively saying what’s happening across the entire platform is impossible, because Facebook remains a gigantic social network with billions of active users. But I have now seen enough to make some educated guesses about what is happening.
Facebook’s Recommendation Algorithm is Injecting AI Into People’s Feeds
On Meta’s first quarter earnings call last week, Mark Zuckerberg said that “right now, about 30 percent of the posts on Facebook feed are delivered by our AI recommendation system. That’s up 2x over the last couple of years.” Zuckerberg is referring to a TikTok-like “For You” system on the News Feed where content is being recommended that is not content from pages or groups that users have specifically liked, nor is it content that people’s friends have interacted with. As Stanford researchers Renee DiResta and Josh A. Goldstein found, AI spam is being recommended using this system and is showing up in people’s feeds. I have seen this on my own feed, but I have also seen it on two brand new, burner Facebook accounts I created to research this story.
On one of those accounts, I specifically sought out AI-generated content and “liked” it and left comments on it. Within a few hours, the vast majority of all content being recommended to me was the same bizarre AI spam that has been repeatedly going viral both on Facebook and off of Facebook.
When I first started reporting on AI spam on Facebook, I had to specifically seek it out by joining groups like “Um, isn’t that AI?,” whose members have been tracking AI spam for the last year. I soon realized that the best way to find bizarre viral spam was by simply logging into this alt account, where my feed was quickly taken over by “Shrimp Jesus,” celebrities who had been altered to have their limbs amputated, images of poor African kids who had built something, AI-generated log cabins, etc. With just a few likes, I was able to turn my feed into one where at least 80 percent percent of the content is AI generated, and where the only content I’m being shown is already incredibly viral elsewhere on the platform.
The news feed on a burner account after liking several AI images
While writing this article, I created another Facebook account with the intention of trying to find out if anyone would try to hack my account if I interacted with AI spam. The first image I saw on Facebook after creating this account was an AI-generated log cabin. Six of the first 10 posts I saw were AI-generated spam, which included an entire page full of AI-altered images of Justin Bieber and Hailey Bieber. One image was of an AI-generated starving, mutilated child asking for birthday wishes. When I clicked through to the page that posted the image, I saw dozens of AI-generated children, some of whom were seemingly dead, drowning, starving, had amputations, or were some combination of all of those. (Facebook deleted these accounts after I wrote an article about them but did not respond to my email requesting comment and an interview about how it tracks and takes action on this type of content). Another image I saw within seconds of joining Facebook was an AI-generated image of Jesus with an amputated leg, wearing an Uncle Sam hat, sitting in a wheelchair and holding a birthday cake while sitting next to an American soldier holding a machine gun in a war zone.
As I began to interact with pages, I was quickly friended by random accounts. I do not think my account has been hacked, but I noticed that the accounts friending me were primarily sharing AI-generated content on their pages and on their Facebook Stories.
The Meta Is Constantly Changing
In online video games, the “meta” is what players generally agree is the best strategy in a given game. Over time, everyone shifts to using this overpowered strategy, creating a new level playing field, until a new meta is discovered. The meta on Facebook is constantly shifting, too.
At first, previously viral (but real) images were being run through image-to-image AI generators to create a variety of different but plausibly believable AI images. These images repeatedly went viral, and seemingly tricked real people into believing they were real. I was able to identify a handful of the “source” or “seed” images that formed the basis for this type of content. Over time, however, most AI images on Facebook have gotten a lot easier to identify as AI and a lot more bizarre. This is presumably happening because people will interact with the images anyway, or the people running these pages have realized they don’t need actual human interaction to go viral on Facebook.
As many people have pointed out, AI images on Facebook have gone from feeling “slightly off” but based on reality to being utterly deranged.
The types of content I have seen go viral have ranged from AI generated log cabins and kitchens and nature scenes and AI-filtered celebrities to shrimp Jesus, Jesus with hot flight attendants, deformed children, deformed celebrities, hot soldiers, African children who “built” something, and variations of all of the above. Some images have begun to use Facebook’s 3D image uploading capabilities, which might help the content perform in the algorithm in some way. In my own internal categorization, I have put images into two categories: “Realistic” AI images and “Bizarre” AI images.
What seems obvious to me is that real people are being fooled by “realistic” AI images, and bots are propping up “bizarre” AI images. Both types are getting injected into real people’s feeds.
Bots and ‘Inauthentic Behavior’
DiResta and Goldstein found using a Facebook analysis tool called CrowdTangle—which Meta is turning off later this year—that at least some of the pages they analyzed for their paper showed obvious evidence of inauthentic behavior and clickfarming. DiResta and Goldstein did a huge amount of their own research which was detailed in their paper, but, because I had already written extensively about AI spam on Facebook, they also dug deeper into a few AI spam pages that I had found.
One of them was a page called Life Nature, which had previously been owned by a metal music tour booking company called Continental Touring which had 9,400 followers. Stephan Mellul, the booking agent who ran the page, told me in December that the page was stolen from him: “No idea how it happened unfortunately, as I was the only admin and my personal profile is still intact.” After the page was hacked and was converted to AI content, DiResta and Goldstein found that it gained 328,000 followers in the week between December 31, 2023 and January 6.
“We’ve seen some pages that get 80,000 followers overnight,” DiResta told me. “You can see demonstrably, there is no universe in which those are real people. That is absolutely, 100 percent a click farm.
Because this page was hacked, and because I reached out to Meta for comment about this specific page, it is one of the few AI-focused pages that I have seen taken down by Facebook. A Facebook spokesperson told me in January that it was investigating this page under its spam/fake account policies, not under any AI policies.
The researchers found that, of the 120 AI spam pages they had analyzed, “46 had changed their names, often from an entirely different subject, and some displayed a massive jump in followers after the name change.”
The phenomenon is too large, however, to be fully explained by bots.
The Zombie Internet
It is reductive to call Facebook the “Dead Internet.” There are real people on Facebook, and real people are being fed this content. The images themselves are being made by AI at the direction of real humans who have learned that spam can be monetized. Real humans at Facebook the company are choosing not to or are not equipped to take action on these accounts or this type of content, which now makes up an unknown but significant portion of content on the site. AI spam, as well as the specter of AI content, is impacting how real people use Facebook and perceive reality more broadly. Facebook itself is shoving its own AI features down people’s throats, and has made clear that it is going to continue spending billions of dollars on AI features that it intends to make core to its products and business model.
The biggest unanswered question from my many months of reporting on this is whether real people are actually being fooled by “shrimp Jesus” and other bizarre AI images, and why they are interacting with this content in the first place.
Real people are interacting with and are being fooled by “realistic” AI content
Here is an AI-altered image from a page called DIY & Crafts, which has 1,800 likes, 713 comments, and 84 shares. A quick glance at this image more or less scans as “real,” because it is a real image that has been taken from a website called Front Porch Ideas and More, which posted it in 2013, and run through an AI image generator to create various AI artifacting (like the garbled text, smoothed knots on the wood planks, and bending on the house’s siding) but not a wholly bizarre image. If you scroll through these comments, most of them are actually engaging with the content of the image.
The AI-altered image on Facebook on the left, and the real image on the right
Comments include things like: “This wood looks awful yellow” and “the middle beam is a goner” and “it looks good so far. I hope you have some kind of waterproof membrane or copper sheet/flashing behind the ledger up under the siding a few inches above and a few inches below the top of the house foundation.” Others say things like “The center joist is not needed. It is taking 1/2 the load of the floor. It is undersized. Size the joists to span to the wall instead and run them from the wall to the outside” and “Hire a real carpenter to fix this. This is very unsafe.” Another comment is “wow, the AI is just in its warm-up stage. To see how many people are taking the time to pick this AI image apart is scary.”
Lots of the comments have back-and-forths between people who do seem to be real because they are actively engaging with each other and discussing the specifics of the actual image that is being shown.
I also clicked through to the profiles of people who are commenting on the images, and they largely appear to be profiles that are controlled by real people, and many of them claim to be contractors or carpenters. What I mean by this is that their Facebook profiles have a variety of different types of activity: They are posting images of their families and are tagging them in the posts, are posting about birthday parties, responding to posts on their walls, and are doing things like posting long updates about their children, a funeral they recently attended, politics, etc. This means that real people are interacting with AI-generated images that are being fed to them by Facebook’s recommendation algorithm, and they do not know that they are not real. In my experience, interacting with this type of content means you will see more of it, and more bizarre forms of it.
It is impossible to calculate an exact number, but given the size of Facebook’s platform and the ubiquity and popularity of AI images on Facebook now, a non-trivial amount of human hours is being wasted thinking, commenting, and giving feedback on AI-altered joists in service of an engagement scheme that converts clicks into dollars via online ads. To be clear, I selected one random photo from one of the thousands of pages that are posting dozens of AI-generated or AI-altered photos every day, and spent a long time finding where the photo originally came from, analyzed the page that was posting it, and clicked through to the profiles of the people commenting on it and analyzed their behavior. This is a microcosm of what is happening constantly on AI-generated images all over Facebook all day every day.
‘Shrimp Jesus’ and bizarre AI is being boosted by bots, but also by real people
This brings me to the “bizarre” AI images of Jesus, flight attendants, starving children, amputees, etc. I have endlessly tried to talk to the accounts who are commenting “Amen” and “happy birthday” and a series of gifs, emojis, etc on AI spam. I have thus far found nothing to suggest that any real portion of the comments on this type of content are “real.” Let’s look at this image of Uncle Sam Amputee Jesus and a soldier with machine gun:
The comments on this are things like “Amen,” “Amin,” “Happy birthday,” “Thank you Lord Amen happy birthday,” etc. On this specific image, the only post that comes from someone who I am sure is a human who is still in control of their own account is from someone who wrote “Same old AI garbage.”
When I clicked through to the profiles of the people who commented on the unfinished deck, I was able to see all sorts of “real” Facebook activity. But when I clicked through to the profiles of people who commented on bizarre AI, very few of them had any recent public activity (some—but very few—of the accounts do follow normal patterns I would associate with human accounts).
This was not universally the case, but lots of the profiles didn’t have any posts I could see from the last few years. If they did have recent activity, it was often just an endless number of memes, videos, and AI content reshared to their walls with no additional comment. I was also able to find a few accounts where, when I searched for the person’s name, I was able to find three or more accounts with the same name and profile picture, suggesting that the person had been locked out of their account and had created a new account. To be clear, I have done this same research on hundreds of photos posted by hundreds of different pages, so my assessments in this section are based on that research and not the comments on any single photo.
I endlessly tried to talk to people who commented on these images, but I had no luck at all. Over the course of several months, I messaged 300 people who commented on bizarre AI-generated images, which I could only do 20 or so at a time before Facebook stopped letting me send messages for several hours. I also commented on dozens of images myself, asking for any human who had also commented on the image to respond to me. Across those hundreds of messages, I got four total responses. One person sent me a thumbs up emoji, then unsent the message. Another person said “your welcome,” one person said “ok” and another, who commented on the specific amputee Jesus Uncle Sam image above, said “Hi Jason. This is my account and this is not fake,” then didn’t answer any follow up questions. Over the years, I have regularly used Facebook messages to contact sources all over the world. While I do not usually have a 100 percent response rate, I get a response most of the time (In December, I was able to get in touch with real artists who were having their content stolen by AI, for example.) When I tried to contact people posting AI spam on LinkedIn last week, I got responses from all five of the people I messaged.
I also posted in the “Um, Isn’t That AI” group and asked if any of the people there had managed to talk to anyone commenting on the posts, because I know many people there have been trying to determine whether these people are “real” or not.
The overwhelming sentiment was that the accounts commenting on these images were also bots: “In my experience, the supermajority of engagement on viral AI Facebook pages is just as artificially-generated as the content they publish. When exploring their comment sections, one will often see hundreds of bot-like comments interspersed with a few ‘real’ people sounding the alarm to no avail,” Khan Schoolcraft, a moderator of the group, told me. “It’s like yelling into an empty room full of a thousand identical brick walls.”
“This is most evident when the content in question is extremely low-quality, comically absurd or offensive in some way,” Schoolcraft added. “Whether it's a child transforming into a water bottle cyborg, a three-armed flight attendant rescuing Tiger Jesus from a muddy plane crash, or a hybrid human-monkey baby being stung to death by giant hornets, all tend to have copy+pasted captions, reactions & comments which usually make no sense in the observed context. It's as if everything from the creation of the images, to the daily management of the accounts, to their fan base and interactions, has been automated to optimize whatever revenue stream this genre helps generate.”
Most of the AI spam is coming from Facebook “Pages,” which are different from Facebook “Groups.” It is possible to “follow” a Page, but you have to “join” a Group. This is a small distinction not necessarily worth getting into, but it is possible to see all of a user’s historical activity within a Group, but it is not possible to see all of a user’s comments on a specific Page. Despite most AI spam coming from “Pages,” there are also AI spam “Groups” that post essentially the same type of content. These Groups make it easy to track user commenting behavior. Within groups that share AI spam, such as “I Love Jesu,” I was able to click on any user who commented on an AI-generated image and see what they commented on other AI-generated images. I regularly saw accounts that just commented the same thing over and over on every image, suggesting they were bots:
Several accounts had activity that made me believe that the account once belonged to a human but doesn’t anymore. A lot of the accounts commenting on this type of content shared authentic-seeming and public-facing posts for years (images of themselves with friends, posts about their life, etc) before suddenly posting either nothing or posting only memes and AI content.
I also found accounts that I think might be being used simultaneously by a real human and a bot. I believe some of these accounts have been compromised, but the passwords have not been changed and so some of their activity is authentic and some of it is not. Specifically, I found a woman whose own Facebook posts—about her job and family, mostly—were exclusively in Greenlandic, but who concurrently commented on AI spam only in English. I also found an account whose wall posts were all in Greek, but who commented on AI spam only in Spanish. My theory is that these accounts have been hijacked to comment and engage with AI spam, but that the owner of the account hasn’t noticed and still has access to the account.
The other thing that I noticed when clicking through to the profiles of accounts commenting “Amen,” “Happy birthday,” or a string of emojis on bizarre AI images is that the overwhelming majority of the accounts engaging with these images have profiles that say they are in countries in the Global South. It is impossible to actually statistically analyze this because Facebook’s platform is so huge, but a huge portion of the comments I saw on this type of content during my research showed that they were from users in Thailand, Vietnam, the Philippines, Cambodia, Nigeria, and Indonesia. Each of these places has its own, specific internet culture, and some of these countries were given free access to Facebook on mobile devices via Facebook’s “Free Basics” project. As a result, some people in these countries’ first interactions with the internet were through Facebook, which disproportionately gave people exposed to the internet in this way a corporatized, Facebook-centric view of the internet. If you know anything about the Free Basics account creation process or anything else that would lead to these accounts being more susceptible to being hijacked, please get in touch.
Free Basics has been long abandoned, but the fact that these countries seem overrepresented in the comments is notable; Facebook allows you to see where a page admin is supposedly located. DiResta and Goldstein of Stanford found that many of the pages they analyzed had admins who were in Vietnam, Cambodia, Thailand, and Armenia. Mong Palatino, a Manila-based activist and internet researcher, told me that a mandatory 2022 SIM registration law in the Philippines was intended to cut down on spam and clickfarms, “but fake accounts and bots remain in place. Phone-related scams persist.”
But I think that we can say that Facebook’s emphasis on injecting “recommendations” into people’s feeds means that Facebook no longer approximates anything like a “social” network. Instead of being a mix of popular text and image posts from people within your network and broader community, it is becoming a feed of content that is popular with people all over the world, who speak a whole lot of different languages. You do not need to speak English, or Thai, or Vietnamese, or Urdu to post an emoji responding to a fucked up Shrimp Jesus. In aggregate, across Facebook’s billions of users, this means purely visual content—AI generated or not—is going to have a larger potential audience than content that can only appeal to people who speak a specific language.
The Poisoning of Social Media and the End of Reality
Even if we assume that the overwhelming majority of AI spam on Facebook is coming from the “Dead Internet” of bots interacting with bots, the fact remains that this bullshit is making its way into real people’s feeds and is poisoning the platform. It has become very difficult to systematically study Facebook, because Facebook is turning off CrowdTangle, a way to track content throughout the network.
As an experiment, though, I posted on my own Facebook for the first time since we launched 404 Media. I asked my friends and family if they had seen AI spam, and to screenshot and send me examples of it. I also asked 404 Media readers to do this in a recent email I sent. People across my networks said that they are seeing AI spam being recommended to them, that they have seen their friends and family interacting with it, and that they are sometimes having a hard time telling what is AI generated and what isn’t.
Many of my Facebook friends sent me screenshots of people they know commenting on clear AI spam, or having it injected into their feeds. But some people also sent me examples of “AI-generated” content that they were mad about having in their feed that was not actually AI-generated.
One person sent me numerous videos of art students sharing their beautiful paintings, which is a common trope of AI-generated spam. But in these cases, the specific images and videos that were sent to me were actually painted by real people who had documented their artistic process. Another person sent me an image of a panda on an airplane, which is “fake” because the panda isn’t real. But the image itself is actually a photograph of a real human and stuffed animal panda, which has gone viral multiple times over the years and is not AI generated or altered. Another sent me real images of real celebrities that hadn’t been altered by AI at all but which they thought might have been because it was posted to a meme page.
Even in the "Um, isn't that AI?" group, people post images that are sometimes not AI, that they think are AI. Many pages I've seen post a mix of AI-generated content and real, low-effort content from elsewhere on the internet. While I am able to tell the difference between AI-generated content and "real" images most of the time, I have come across some photos that I'm not sure about, which I am sure will become more common as AI image generators improve.
“At least in some cases, we see a bunch of comments from what look like real people saying ‘Congratulations on your beautiful art,’ indicating that they are not realizing that this is fake,” Stanford’s Goldstein said. “I’m more concerned about people losing trust in their own ability to determine what’s real than I am about any one random image gaining traction.”
All of this, taken together, is why I think we should not view Facebook’s AI spam through the lens of the “Dead Internet.” The platform has become something worse than bots talking to bots. It is bots talking to bots, bots talking to bots at the direction of humans, humans talking to humans, humans talking to bots, humans arguing about a fake thing made by a bot, humans talking to no one without knowing it, hijacked human accounts turned into bots, humans worried that the other humans they’re talking to are bots, hybrid human/bot accounts, the end of a shared reality, and, at the center of all of this: One of the most valuable companies on the planet enabling this shitshow because its human executives and shareholders have too much money riding on the mass adoption of a reality-breaking technology to do anything about it.
