Wikipedia has seen their traffic surge by 50%. And this is just since January of 2024 last year.

What is the massive surge in usage? You might ask, oh, maybe they're getting a ton of new user.

Maybe everyone's like sick of chat GPT so they want to go over to Wikipedia.

Wrong. This is all due to AI models and AI scrapers crawling their website for information and driving up the cost of Wikipedia a ton.

So today on the podcast, I want to dive into this phenomenon, but not just because of Wikipedia, while it's interesting how it kind of affects one of the biggest websites on the planet, it's because of how it's going to affect every single website on the planet, every single

business, every single person that has anything online is going to have this exact same

problem.

And some of the solutions are actually pretty hilarious, but let's get into this.

The first thing I wanted to say is an official statement that Wikipedia published on their

official blog detailing a little bit of this problem and pretty much what's happening.

They said, our infrastructure is built to sustain sudden spikes from humans during high

interest events. But the amount of traffic generated by scraper bots is unprecedented and presents

growing risks and costs. So the thing that's really interesting here is, yes, Wikipedia

is free for anyone to use and

technically even AI models to scrape.

It's kind of just how it was built, right?

It's not like they have a big, you know, team of journalists that go and write articles.

Anyone can contribute.

And so it's kind of like fair, fair game for anyone to use this content.

But the problem is that these AI models are using the content.

And the bigger problem is that even if a website and Wikipedia in this case doesn't actually

do this because they want to be indexed by Google, but even if a website use a robot.txte

...