AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt

[email protected]

what is your deal?

[email protected]

I liked it back when link aggregators were the go-to for discovery. You could have sites that were real gems that were just tucked away.

I think the indexing started out ok. Counting backlinks and using that as a ranking was pretty genius, right up until people realized they could game the system, then google realized that artificially screwing with their own system was worth money, then the used ads to modify ranking.

ads to modify discoverability the death of free internet

[email protected]

The only AI company that responded to Ars' request to comment was OpenAI, whose spokesperson confirmed that OpenAI is already working on a way to fight tarpitting.

Ah yes. It extremely common for one of the top companies in an industry to spitefully expend resources fighting the efforts of...

One or two people

Please, continue to grace us with you unbiased wisdom. Clearly you've read the article and aren't just trying to simp for AI or start flame wars like a petulant child.

[email protected]

Not like you can load balance requests of the malicious subdirectories to a non-prod hardware. Can be decommissioned hardware.

[email protected]

Well, luckily for them, it's a pretty simple fix. Congrats on being a part of making them jot down a note to prevent tarpitting when they get around to it. You've saved the internet!

And stop pretending like you're unbiased either. We both have our preconceived notions, and you're not more likely to be open to change yours than I am. In fact, given the hysterical hyperventilating anti-AI "activists" get to, we both know you're not ever going to change your mind on AI, and as such you'll glom onto any small action you think is gonna stick it to the man, no matter whether that action is going to have any practical effect on the push for AI or not.

[email protected]

I'm just vibing, watching the hysterics you guys get up to.

[email protected]

No you’re being a petulant, naysaying child. Leave us alone and go play with your duplos. Adults are talking.

[email protected]

How many hobby website admins have load balancing for their small sites? How many have decommissioned hardware? Because if you find me a corporation wiling to accept the liability doing something like this could open them up to, I'll pay you a million dollars.

[email protected]

Bigotry? From a lemmy user? Never seen it before!

If you don't like what I'm saying, block me and move along. Or report my comments, if you think they're offensive enough. If I'm breaking a rule or the mods don't like what I have to say, maybe they'll remove them, or even ban me from the comm! That's the limit of your options for getting rid of me though.

[email protected]

Interesting. Mega supporters are now cold-blooded.

[email protected]

Bigotry lmao talk about a Hail Mary.

[email protected]

I get that the Internet doesn't contain an infinite number of domains. Max visits to a each one can be limited. Hel-lo, McFly?

agnos.is Forums

AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt