Anubis - Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers
-
I just started using this myself, seems pretty great so far!
Clearly doesn't stop all AI crawlers, but a significantly large chunk of them.
Anubis is provided to the public for free in order to help advance the common good. In return, we ask (but not demand, these are words on the internet, not word of law) that you not remove the Anubis character from your deployment.
If you want to run an unbranded or white-label version of Anubis, please contact Xe to arrange a contract.This is icky to me. Cool idea, but this is weird.
-
Anubis is provided to the public for free in order to help advance the common good. In return, we ask (but not demand, these are words on the internet, not word of law) that you not remove the Anubis character from your deployment.
If you want to run an unbranded or white-label version of Anubis, please contact Xe to arrange a contract.This is icky to me. Cool idea, but this is weird.
...Why? It's just telling companies they can get support + white-labeling for a fee, and asking you keep their silly little character in a tongue-and-cheek manner.
Just like they say, you can modify the code and remove for free if you really want, they're not forbidding you from doing so or anything -
...Why? It's just telling companies they can get support + white-labeling for a fee, and asking you keep their silly little character in a tongue-and-cheek manner.
Just like they say, you can modify the code and remove for free if you really want, they're not forbidding you from doing so or anythingYeah, it seems entirely optional. It's not like manually removing the Anubis character will revoke your access to the code. However, I still do find it a bit weird that they're asking for that.
I just can't imagine most companies implementing Anubis and keeping the character or paying for the service, given that it's open source. It's just unprofessional for the first impression of a company's website being the Anubis devs' manga OC...
-
…you do realize that brute forcing it is the work you use to prove yourself, right? That’s the whole point of PoW
True, I should of phrased that better.
The issue is that sha256 is fairly easy to do at scale. Modern high performance hardware is well optimized for it so you could still perform attack with a bunch of GPUs. AI scrapers tend to have a lot of those.
-
Giant middle finger from me -- and probably everyone else who uses NoScript -- for trying to enshittify what's left of the good parts of the web.
Seriously, FUCK THAT.
You should blame the big tech giants and their callous disregard for everyone else for the Enshittification, not the folks just trying to keep their servers up.
-
Yeah, it seems entirely optional. It's not like manually removing the Anubis character will revoke your access to the code. However, I still do find it a bit weird that they're asking for that.
I just can't imagine most companies implementing Anubis and keeping the character or paying for the service, given that it's open source. It's just unprofessional for the first impression of a company's website being the Anubis devs' manga OC...
It is very.. outspoken yes, but this is just their branding. Their blog is full of anime characters like that.
And it's not like you're looking at a literal ad for their company or with their name on it. In that sense it is subtle, though a bit unusual.
-
Meaning it wastes time and power such that it gets expensive on a large scale? Or does it mine crypto?
-
It is very.. outspoken yes, but this is just their branding. Their blog is full of anime characters like that.
And it's not like you're looking at a literal ad for their company or with their name on it. In that sense it is subtle, though a bit unusual.
I don't think it's necessarily a bad thing. Subtle but unusual is a good way to describe it.
However, I would like to point out that if it is their branding, then the character appearing is an advertisement for the service. It's just not very conventional or effective advertising, but they're not making money from a vast majority of implementations, so it's not very egregious anyway.
-
I just started using this myself, seems pretty great so far!
Clearly doesn't stop all AI crawlers, but a significantly large chunk of them.
I not find any instruction on the source page on how to actually deploy this. That would be a nice touch imho.
-
It's a rather brilliant idea really, but when you consider the environmental implications of forcing web requests to ensure proof of work to function, this effectively burns a more coal for every site that implements it.
I don't think AI companies care, and I wholeheartedly support any and all FOSS projects using PoW when serving their websites. I'd rather have that than have them go down
-
It is isn't on many levels.
-
It only runs against the Firefox user agent. This is not great as the user agent can easy be changed. It may work now but tomorrow that could all change.
-
It doesn't measure load so even if your website has only a few people accessing it they will stick have to do the proof of work.
-
The POW algorithm is not well designed and requires a lot of compute on the server which means that it could be used as a denial of service attack vector. It also uses sha256 which isn't optimized for a proof of work type calculation and can be brute forced pretty easily with hardware.
In summary the Tor implementation is a lot better. I would love to see someone port it to the clearnet.
I look forward to TOR's PoW coming out for FOSS WAFs
-
-
I not find any instruction on the source page on how to actually deploy this. That would be a nice touch imho.
There are some detailed instructions on the docs site, tho I agree it'd be nice to have in the readme, too.
Sounds like the dev was not expecting this much interest for the project out of nowhere so there will def be gaps.
-
I just started using this myself, seems pretty great so far!
Clearly doesn't stop all AI crawlers, but a significantly large chunk of them.
I think the maze approach is better, this seems like it hurts valid users if the web more than a company would be.
-
...Why? It's just telling companies they can get support + white-labeling for a fee, and asking you keep their silly little character in a tongue-and-cheek manner.
Just like they say, you can modify the code and remove for free if you really want, they're not forbidding you from doing so or anythingJust like they say, you can modify the code and remove for free if you really want, they’re not forbidding you from doing so or anything
True, but I think you are discounting the risk that the actual god Anubis will take displeasure at such an act, potentially dooming one's real life soul.
-
I just started using this myself, seems pretty great so far!
Clearly doesn't stop all AI crawlers, but a significantly large chunk of them.
Nice. Crypto miners disguised as anti-AI.
-
Nice. Crypto miners disguised as anti-AI.
what about this is crypto mining?
-
I just started using this myself, seems pretty great so far!
Clearly doesn't stop all AI crawlers, but a significantly large chunk of them.
-
It's a clever solution but I did see one recently that IMO was more elegant for noscript users. I can't remember the name but it would create a dummy link that human users won't touch, but webcrawlers will naturally navigate into, but then generates an infinitely deep tree of super basic HTML to force bots into endlessly trawling a cheap-to-serve portion of your webserver instead of something heavier.
generates an infinitely deep tree
Wouldn't the bot simply limit the depth of it's seek?
-
I just started using this myself, seems pretty great so far!
Clearly doesn't stop all AI crawlers, but a significantly large chunk of them.
Why Sha256? Literally every processor has a crypto accelerator and will easily pass. And datacenter servers have beefy server CPUs. This is only effective against no-JS scrapers.
-
I think the maze approach is better, this seems like it hurts valid users if the web more than a company would be.
For those not aware, nepenthese is an example for the above mentioned approach !
