How Fake AI Political Accounts Fool Us

AI-generated “personalities” are no longer just a glimpse of the future—they’re here, speaking directly to us, blending seamlessly into our social media feeds, and sometimes racking up huge followings before being exposed as fake. As these synthetic figures become more convincing, the bigger question isn’t just how they’re created, but why people believe them.

A recent viral TikTok account known as “MAGA Megan” offers a perfect case study in how AI-driven misinformation intersects with human psychology, identity, and the incentives of social media platforms. This “MAGA Megan” account has racked up hundreds of thousands of views on TikTok, but as Hank Green recently pointed out, it definitely appears to be an AI-generated fake.

The account featured a woman speaking over music (likely to distract away from her speech), hitting all the culture war buzzwords, and saying exactly what a certain segment of viewers wants to hear. But if you look and listen closely, her speech patterns are off and there are no signs of a real human presence (e.g., no clutter in the background, no stumbles, no signs of improvisation, etc). The account was reported as being fake and was eventually banned.

Before that happened, the YouTuber Parkrose Permaculture pointed out how the account would try and get people to download their app in between the political outrage bait. It’s all a synthetic creation built to go viral and get people to click on whatever they are selling on their website.

But why do so many people believe it's real? I think there are three main reasons:

Identity Over Authenticity

As I discuss in my new book Misguided, people are motivated to believe what aligns with their identities, which is not necessarily what’s true. If a video reinforces your values, grievances, or worldview, you’re more likely to accept it at face value. Whether it’s coming from a real person or a fake one doesn’t matter as much; what matters is that it feels right.

AI-generated content is getting better at mimicking this feeling that takes advantage of our identity biases. When something confirms our worldview, we are much less likely to think critically and pause to verify the source. I also described how this process applies to AI deepfakes surrounding wars and conflicts in my interview with the BBC.

The Power of the Network

When fake accounts are shared by people you trust—or simply go viral within your community—they gain credibility through repetition and familiarity. TikTok’s algorithm doesn’t care whether the account is real; it promotes content that keeps people watching. The result? A synthetic identity can be woven into real human networks, spreading like wildfire.

This is a great example of what I describe in both my research and my book: misinformation can be shared through mutually reinforcing spirals through our networks on social media. Fake accounts don’t need to be convincing on their own; they just need to enter the right echo chamber, where they can be reinforced and provide social proof as being real enough to interact with. More broadly, a big reason why people believe fake AI accounts is the same as the reason they fall for other forms of misinformation: It’s not just about the information, it’s about belonging.

Platform Incentives and Algorithmic Blind Spots

Social media platforms like TikTok aren’t optimized for truth; they’re optimized for engagement. The more inflammatory or emotional a video is, the more likely it is to spread. That creates fertile ground for fake AI personas designed to provoke and divide. Fake accounts can also promote each other, boosting their engagement and pushing them into your timeline.

Artificial Intelligence Essential Reads

How Children's Brains Pay for AI Twice

The Horrors of the Empty Office Building in 'Backrooms'

TikTok, YouTube Shorts, and Instagram Reels aren’t well-equipped to distinguish AI content from real people, especially when creators actively try to deceive. And because these videos can be created at scale, it’s easier than ever to flood the feed with synthetic personalities targeting every niche.

What Can We Do About It?

Combating misinformation takes more than just fact-checking; it requires stronger education for both media literacy and awareness of our own biases. Some of my research shows promising results in helping people quickly identify manipulated images online. But we’ll remain vulnerable as long as our identity biases drive our engagement. To build real resilience, we need to broaden our social networks and diversify our relationships so we’re not so motivated to defend our political group that we blindly trust any content that aligns with our side. I also hope that platforms will take these AI fake accounts more seriously and at least demonetize the worst of AI slop.

That “MAGA Megan” TikTok account might be fake, but the trust people place in her is real. If we want to defend against synthetic manipulation, we need to understand not just what people believe, but why it resonates. As AI evolves and plays a bigger role in shaping what we see, understanding our own biases becomes even more critical.

This post also appears on Misguided: The Newsletter.