Everyone’s buzzing about how AI is revolutionizing the way we search, but here’s the surprise: when it comes to actually directing visitors to sites, Google still dominates the internet, and AI systems hardly leave a mark.
As per TollBit’s State of the Bots Q2 2025 report, Google directs 831 times more traffic to publishers than all AI systems put together. While AI bots are scraping the web like their life depends upon it, they’re sending very few actual humans back.
Scraping More, Sending Less AI tools are scrapiling every nook and cranny of the web, but not paying publishers for it.
The study discovered that AI applications contributed only 0.102% of overall referrals, while Google alone forwarded an impressive 84.1% of all external traffic. In short, for every single referral visitor by an AI system, Google referred eight hundred thirty-one. And it’s getting worse. On average, 135 AI scrapes were needed to yield a single human referral. That is to say, publishers are paying for bandwidth and server resources and receiving near zero in return. Bots Are Rising, Humans are falling AI bots have become an increasingly large percentage of the internet’s traffic rate.
Early this year, 1 in 200 visits was from an AI bot. By Q2 2025, it’s 1 in 50: a quadrupling of just six months. While human visitors declined 9.4%, this trend keeps media and content companies up at night. AI crawlers have also eclipsed Microsoft Bingbot in raw crawl volume, an indication of a seismic shift in web use or misuse.
Even Google is not exempt. Following the introduction of AI Overviews, Googlebot crawling activity increased 34.8%, yet referral clicks dropped by almost 25%. More crawling. Fewer clicks. Alteration of Content Habits: AI technologies are reprogramming the way content is being consumed. Rather than clicking through to websites, users are receiving summarized responses in chat interfaces, bypassing publishers altogether.
The TollBit statistics uncover some interesting facts:
- B2B and professional content experienced the largest scraping-to-human ratio. Parenting (up 333%) and shopping/deals (up 111%) experienced the most rapid AI request increases.
- National news had 5x as many real-time AI scrapes as model-training crawlers. And geographically, APAC publishers were affected most, with 3x as many AI requests as U.S. sites.
- Europe declined 27%, perhaps as a result of more stringent data and privacy legislation. The Bot War Has Started. Publishers are not kidding around. The report reveals a 336% increase in sites blocking AI bots and a 360% increase in “Bot Paywall” hits systems that pay crawlers for entrance. Still, many AI bots don’t play by the rules. 13% of them ignored robots.txt restrictions in Q2 2025, up from 3.3% the previous quarter.
- OpenAI’s 404 errors jumped from 0.3% to 3.7%, often due to hallucinated URLs. • Anthropic’s Claude, however, improved dramatically reducing its error rate from 55% to 4.8% after gaining live web access.
Why It Matters: The web we know today is under threat. For decades, there was a reasonable balance between publishers giving away search content and search engines returning traffic. However, with AI applications now scraping more and returning less, that equilibrium is shattering. As more content is absorbed by AI models and fewer users actually visit websites, publishers face higher costs, fewer readers, and shrinking revenue. That’s why we’re seeing experiments like Cloudflare’s pay-per-crawl and RSL’s “Bot Licensing” models emerge. If this pattern continues, the open web could become a closed loop dominated by AI systems that feed on content but rarely feed it back.
The Bottom Line: AI may be winning the crawl race, but Google is still the reigning click king. For every 831 individuals that Google directs, AI directs only one. Until then, the internet’s lifeline continues to be what it has always been: human clicks, not bot scrapes.
