If Anubis is so great, why isn't it everywhere?

[argumentativemonotheist@lemmy.world]

People get sensitive when religion is mentioned!

[underline960@sh.itjust.works]

My guess is it's the tone/wording that implies that OP doesn't think it's great.

I agree that it's a legit question, though.

[kbal@fedia.io]

It isn't so great, and it is everywhere.

[renegadespork@lemmy.jelliefrontier.net]

Yeah, that’s the dilemma. The more popular it gets, the more crawlers will be designed to circumvent it.

[ephera@lemmy.ml]

I believe, (far too) much of the commercial world relies on Cloudflare to solve that problem.

And as for Wikipedia, any AI trainer worth their salt should know that they don't need to crawl it, because you can actually just download the whole Wikipedia dataset.

[stringere@sh.itjust.works]

Set is so much cooler.

[traches@sh.itjust.works]

• not every website has the problem it solves
• not everyone who does likes the solution it offers
• web development moves fast but not that fast

[zloubida@sh.itjust.works]

Pffff stop with your superstitions. Anybody serious knows that Teutates is the best.

[shadow@lemmy.ca]

It's not great, it's actually pretty ineffective.

https://lock.cmpxchg8b.com/anubis.html

[luffy879@lemmy.ml]

10 websites have deployed.

Source?

Because many already have solved the Problem via cloudflare, etc.

[icystar@lemmy.cif.su]

It's not that great, and the creator seems to have some weird fixation on little girls.

[icystar@lemmy.cif.su]

I think we're probably going to see a schism at some point in the fediverse.

The fragile people who can't seem to tolerate anything they don't like will end up in their own bubble.