After about five months of AI bots hungrily munching on my blog’s free training data, they seem to have abruptly stopped. It’s a good sign, but given they seem to be hammering most other sites on the web, it’s somewhat odd. Here, I have a quick look at the numbers to see what’s been happening.
Rising views
Although I removed analytics collection from my blog years ago, Hetzner (where my blog is hosted) generates basic statistics from their web server logs,1 and sometimes I have a nosey to see if anything interesting is happening. Stats derived from these web server logs have limited information content, which is how I like it. This way, I get rough information about site visits while not tracking anyone.
Late last year, I noticed an upwards trend in the number of hits to my blog. I observed that there were many more hits than usual and that the number rose steadily over the following few months. At the end of the year, I wasn’t surprised to see names in the “Robots/Spiders” section from a well-known AI company. I thought to myself, “Oh ok, that’s just a horde of AI bots sucking data from my blog, much like they’re doing to everyone else”. In other words, “Move along. Nothing to see here. This is the new normal”. Not having the time to investigate further or take any mitigating measures, I carried on with my life.
A sudden drop
That was the situation until late March, when the numbers returned to “normal” levels. For my tiny site, it’s quite the drop-off. You can see numbers begin their decline from the 19th until the 24th of March:
I mean, say what? Don’t the bots like my yummy free training data anymore?
To be honest, I don’t have an explanation for the behaviour, but I thought it’d be fun to dig into the numbers a little and see what’s happening. I’m wondering if the trend will continue into April and the rest of the year.
Xmas lift-off
Anyway, the story begins in earnest in December last year, when I happened to notice a jump in the number of visits to my blog. To some degree, that can be a flattering indicator: people like my articles! Yay!
Look at the number of different visitors (“Unterschiedliche Besucher”) ramping up over the course of 2025:
Then, at the end of December,2 the hits per day stats (“Anzahl der Besuche”) reached a high level3 (~1,700 hits per day),
eventually touching the 3,000 hits per day mark later, in March 2026.
Unfortunately, a quick look at other data from December 2025 showed lots of hits from bots:
That was somewhat deflating. Ok, my articles aren’t being read by humans and aren’t that useful to people. 😕 Instead, bots are gobbling them up and are (probably) dishing them up in a mangled form in someone’s search results. That’s quite the bummer.
Hitting the categories for no good reason
Digging into the stats a little more, it turned out that the daily bot-snacking began in roughly October 2025. Yet, definitive evidence only turned up in the “Robots/Spiders” stats later.
What gave their presence away? Well, the “categories” page was being downloaded so often that it appeared in the list of top 10 pages.
Now, the “categories” page is a pretty bloody boring page. It’s not something a human is going to look at much, and hence, this page shouldn’t appear in the top 10 list. I mean, it’s not often you want to know what categories some blogger uses on their page, right?
From October 2025 onwards, the numbers start to become very weird, at least if one expects the traffic to be coming from humans. To get a feel for what I mean, consider that in September 2025, the “categories” page didn’t appear in the top 10. In October, it suddenly appeared at the fourth position in the top 10 pages with 499 hits. That was the first sign that something odd was happening. In November, it was 700 hits. Then, in the following months, 17,538, 34,798, and 32,631 hits respectively, reaching 39,997 hits in March 2026.
Like, WTAF? Just why??? Why don’t they hit the other pages as well?
Maybe they are. But why hammer the categories page?
It’s difficult to know how many of the hits to the remaining pages are from actual human beings, if at all. After all, it’s nice to know if an article has been popular. This gives me an indication of the kinds of posts that are useful and/or interesting to people. If some topic resonates, then I can consider writing more along such lines in the future.
The stats for the top 10 countries tell a similar story. For instance, the United States had ~8,000 hits and ~700 MB, respectively, in September 2025. These stats jump to ~24,500 hits and 5.6 GB in December, achieving ~50,000 hits and 12 GB in March 2026. That’s pretty nuts for such a small site, and it washes out any useful signal that might be in the data.
Direct AI bot evidence
The first direct evidence of AI bots scraping the site appeared in the “Robots/Spiders” stats in November 2025, even if the site was already being regularly scraped before then.
That’s when the bot names ChatGPT-User (OpenAI) and OAI-SearchBot appeared in the top 10 list. From January onwards, they were the top two, which was the same month that Google’s crawler got pushed off the top 10 list. I find it amazing that Google’s bot got shouldered out of the top 10. Did Google’s bots get more efficient and hence no longer need to access the site as often? No idea.
(OpenAI)
More questions than answers
Anyway, it seems the bots have now lost interest to a certain degree. Which is good, I guess. But it does leave me wondering why. Has someone realised that it’s not necessary to poll the site many tens of thousands of times a month when I only post weekly? I don’t reckon. Have they run out of money? Not likely.
I’ve wondered if the drop off is due to some kind of measure implemented by Hetzner to protect small sites like mine. I couldn’t find anything in the Hetzner docs about such measures, so that reason seem improbable right now. It’s hard to know the actual cause for the drop in traffic and the sudden bot disinterest.
I enjoy sharing what I know and things I’ve learned with others. Thus, I sometimes wonder if an increase in visits comes from more people finding the articles appealing. That would be great! After all, I write articles because some problem or topic was interesting to me, hence I’d like to think what I’ve written is also of interest or is useful to others. But with so much “attention” coming from the AI bots, there’s no way to tell how many hits are coming from real human beings. The stats have (or, at least, had) become, for all practical purposes, worthless.
Perhaps now that the bots have lost interest, the numbers will one day reflect actual human usage and hence actual value for people. I’ll have to wait a while to find out.
I’m curious to see if the bots leave me alone long-term. That would be nice. Time will tell!
Hetzner uses AWStats and ReportMagic to generate traffic statistics from the Apache logs of simple static sites. In this article, I’ve focused only on the AWStats output. ↩
Oddly enough, this happened on Boxing Day after very little activity on Christmas Eve and Christmas Day. Do AI bots take a break for Christmas? ↩
I say “high”, but to be honest, this is relative. 1,700 hits a day is nothing for many sites. But when 300-400 hits per day is the norm, this is quite the difference. ↩
Top comments (0)