Kernel.org maintainer says AI crawlers now consume more CPU than all legitimate Git access combined

A Kernel.org maintainer says AI crawlers are creating constant server load by rendering Git commits as HTML instead of cloning repositories directly. Across five geo-distributed nodes, about 14 CPU cores are reportedly dedicated at any time to serving scraper traffic.

The post highlights how demand for clean pre-AI training data is shifting infrastructure costs onto open source projects. If this pattern continues, more public repositories may adopt crawler controls, rate limits, or preferred bulk-access mechanisms to protect shared resources.