Bots Dominate Digital Highways as Nearly 30% of Web Traffic Goes Non-Human

The digital world is experiencing its own rush hour — except the commuters aren’t human. Automated bots now account for nearly 30% of all web traffic as AI companies aggressively deploy crawling operations to feed their hungry language models. The result is an internet under pressure, creaking much like it does during a crisis.

• MetaMETA
  leads the crawler stampede at 52% of observed automated traffic, while GoogleGOOGL
  captures 23% and OpenAI handles 20% of the digital harvesting.
• Commerce, media, and entertainment sectors face the most intense scraping, with crawler activity making up nearly 80% of total AI bot volume across these industries.

The infrastructure awakening: Edge computing platforms like CloudflareNET

and FastlyFSLY

have emerged as unexpected winners, with Cloudflare nearly tripling over the past year as companies scramble for bot management solutions. The problem isn’t always malicious — one overeager fetcher bot recently hammered a site with 39K requests per minute, inadvertently mimicking an attack. And with OpenAI’s GPTBot now indexing roughly 95% of observed domains, website operators are implementing layered defenses, including robots.txt rules, rate limits, and licensing platforms like Tollbit to transform resource drains into potential revenue streams.