Sysadmins and LLM crawlers

The crawlers that collect data to train LLMs on cost sysadmins lots of time, writes Drew DeVault:

instead of working on our priorities at SourceHut, I have spent anywhere from 20-100% of my time in any given week mitigating hyper-aggressive LLM crawlers at scale

(From: Please stop externalizing your costs directly into my face)

Hidde de Vries (@hdv@front-end.social) is a web enthusiast and accessibility specialist from Rotterdam, The Netherlands. He currently works with the NL Design System team and is a participant in the Open UI Community Group. Previously, he worked for W3C (WAI), Mozilla, the Dutch government and others as a freelancer. Hidde spoke at 79 events, most recently in Bristol, UK.

Buy me a coffee Follow @hdv on Mastodon