A new report by Tollbit reveals that the click-through rates for AI chat bots to publisher sites are 95.7% lower than traditional Google search, with a referral rate of just 0.37%. It’s what we’ve known – and what we’ve been warning of for years.
This shines a light on the damage being done by the illegal scraping and repackaging of publishers' content and what effectively is a firewall that has been set up between us and our audiences.
More key findings from the report:
AI bot scrapes that bypassed robots.txt grew by over 40% between Q3 to Q4.
Blocking AI bots via robots.txt remains an insufficient mechanism to prevent unwanted scraping.
The pace of scraping growth accelerated; scrapes per website doubled from Q3 to Q4 and scrapes per page more than tripled.
Even when blocked, some apps continue to access sites through unidentified user agents. Unidentified user agent scraping was roughly equivalent to identified AI bot scraping Q4.
As a result of these illegal practices, publishers are losing subscription and advertising revenue and this impacts our ability to create the independent professional content that AI companies rely on. We need a responsible AI sector.
Everyone loses out at the moment: the AI companies, consumers and the companies funding the professional fact-checked content that is essential to democratic society.