Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

agnos.is Forums

  1. Home
  2. Technology
  3. Cloudflare announces AI Labyrinth, which uses AI-generated content to confuse and waste the resources of AI Crawlers and bots that ignore “no crawl” directives.

Cloudflare announces AI Labyrinth, which uses AI-generated content to confuse and waste the resources of AI Crawlers and bots that ignore “no crawl” directives.

Scheduled Pinned Locked Moved Technology
technology
217 Posts 129 Posters 1.3k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • buelldozer@lemmy.todayB [email protected]

    and try to slam your site with like 200+ requests per second

    Your solution would do nothing to stop the crawlers that are operating 10ish rps. There's ones out there operating at a mere 2rps but when multiple companies are doing it at the same time 24x7x365 it adds up.

    Some incredibly talented people have been battling this since last year and your solution has been tried multiple times. It's not effective in all instances and can require a LOT of manual intervention and SysAdmin time.

    https://thelibre.news/foss-infrastructure-is-under-attack-by-ai-companies/

    ? Offline
    ? Offline
    Guest
    wrote on last edited by
    #105

    Yep. After you ban all the easy to spot ones you're still left with far too many hard to ID bots. At least if your site is popular and large.

    1 Reply Last reply
    0
    • D [email protected]

      the used market

      ? Offline
      ? Offline
      Guest
      wrote on last edited by
      #106

      so you're saying it's a niche toy you can get if you really want one, but nobody's pushing that shit on you, and if you never want to talk about one again in your life, you can probably do that?

      I would like if large language models were in this position. I don't think you understand the degree to which our productive capacity and infrastructure are committed to this technology. it's a lot. basically all the cutting edge computer chips being made are specialist chips for processing large (whatever) models, including the engineering to back it. that means everything else is bumped down a generation or five.

      then there's the amount of electricity being put towards these things-we are in a climate disaster, we do not have green energy, and these things are drawing in the high GW/low TW of energy.

      water is also a lot more precarious than a lot of people want to think about. these things are using lots and lots of good drinkable water to cool those specialist chips, then just being throws out, because I assume it's cheaper than cooling the hot water back down.

      if these things didn't ruin the internet, and were just something you ran on the beefy desktop you otherwise use for games, I wouldn't mind, but that's not where we are.

      1 Reply Last reply
      0
      • tea@programming.devT [email protected]
        This post did not contain any content.
        ? Offline
        ? Offline
        Guest
        wrote on last edited by
        #107

        And soon, the already AI-flooded net will be filled with so much nonsense that it becomes impossible for anyone to get some real work done. Sigh.

        C 1 Reply Last reply
        0
        • H [email protected]

          My dude, they'll literally sell services to both sides of the market.

          A This user is from outside of this forum
          A This user is from outside of this forum
          [email protected]
          wrote on last edited by
          #108

          1 Reply Last reply
          0
          • L [email protected]

            The article says it's not poisoning the AI data, only providing valid facts. The scraper still gets content, just not the content it was aiming for.

            ? Offline
            ? Offline
            Guest
            wrote on last edited by
            #109

            if you're dumb enough to trust a large language model because someone told you "iTs Ai!" no amount of facts will be of great utility to you.

            X 1 Reply Last reply
            0
            • D [email protected]

              On on hand, yes. On the other...imagine frustration of management of companies making and selling AI services. This is such a sweet thing to imagine.

              M This user is from outside of this forum
              M This user is from outside of this forum
              [email protected]
              wrote on last edited by
              #110

              I just want to keep using uncensored AI that answers my questions. Why is this a good thing?

              cilethesane@lemmy.caC explodicle@sh.itjust.worksE 2 Replies Last reply
              0
              • tea@programming.devT [email protected]
                This post did not contain any content.
                ? Offline
                ? Offline
                Guest
                wrote on last edited by
                #111

                You have Thirteen hours in which to solve this labyrinth before your baby AI becomes one of us, forever.

                C 1 Reply Last reply
                0
                • tea@programming.devT [email protected]
                  This post did not contain any content.
                  K This user is from outside of this forum
                  K This user is from outside of this forum
                  [email protected]
                  wrote on last edited by
                  #112

                  Burning 29 acres of rainforest a day to do nothing

                  C ? 2 Replies Last reply
                  0
                  • ? Guest

                    cool, but where do you get them? you can't, right? because they were stupid?

                    S This user is from outside of this forum
                    S This user is from outside of this forum
                    [email protected]
                    wrote on last edited by
                    #113

                    You can still buy new ones. Take a nap.

                    ? 1 Reply Last reply
                    0
                    • tea@programming.devT [email protected]
                      This post did not contain any content.
                      D This user is from outside of this forum
                      D This user is from outside of this forum
                      [email protected]
                      wrote on last edited by
                      #114

                      Considering how many false positives Cloudflare serves i see nothing but misery coming from this.

                      dave@lemmy.nzD X 2 Replies Last reply
                      0
                      • ? Guest

                        Why do I have the feeling that I will end up in that nightmare with my privacy focused and ad-free Browser setup. I already end up in captcha hell too often because of it.

                        D This user is from outside of this forum
                        D This user is from outside of this forum
                        [email protected]
                        wrote on last edited by
                        #115

                        Of course it will. Cloudflare has already ruined the web and it's just another step further.

                        1 Reply Last reply
                        0
                        • R [email protected]

                          this is some fucking stupid situation, we somewhat got a faster internet and these bots messing each other are hugging the bandwidth.

                          D This user is from outside of this forum
                          D This user is from outside of this forum
                          [email protected]
                          wrote on last edited by
                          #116

                          Lol website traffic accounts for like 1% of bandwidth budget. 1 netflix movie is like 20k web pages.

                          1 Reply Last reply
                          0
                          • F [email protected]

                            Cloudflare kind of real for this. I love it.

                            It makes perfect sense for them as a business, infinite automated traffic equals infinite costs and lower server stability, but at the same time how often do giant tech companies do things that make sense these days?

                            D This user is from outside of this forum
                            D This user is from outside of this forum
                            [email protected]
                            wrote on last edited by
                            #117

                            I'm sorry how does it make sense?

                            1 Reply Last reply
                            0
                            • D [email protected]

                              Considering how many false positives Cloudflare serves i see nothing but misery coming from this.

                              dave@lemmy.nzD This user is from outside of this forum
                              dave@lemmy.nzD This user is from outside of this forum
                              [email protected]
                              wrote on last edited by
                              #118

                              In terms of Lemmy instances, if your instance is behind cloudflare and you turn on AI protection, federation breaks. So their tools are not very helpful for fighting the AI scraping.

                              appoxo@lemmy.dbzer0.comA 1 Reply Last reply
                              0
                              • S [email protected]

                                You can still buy new ones. Take a nap.

                                ? Offline
                                ? Offline
                                Guest
                                wrote on last edited by
                                #119

                                somehow I get the impression all this defense of tomogatchis is not about tomogatchis.

                                1 Reply Last reply
                                0
                                • M [email protected]

                                  I just want to keep using uncensored AI that answers my questions. Why is this a good thing?

                                  cilethesane@lemmy.caC This user is from outside of this forum
                                  cilethesane@lemmy.caC This user is from outside of this forum
                                  [email protected]
                                  wrote on last edited by
                                  #120

                                  Because it's not AI, it's LLMs, and all LLMs do is guess what word most likely comes next in a sentence. That's why they are terrible at answering questions and do things like suggest adding glue to the cheese on your pizza because somewhere in the training data some idiot said that.

                                  The training data for LLMs come from the internet, and the internet is full of idiots.

                                  L M 2 Replies Last reply
                                  0
                                  • M [email protected]

                                    I just want to keep using uncensored AI that answers my questions. Why is this a good thing?

                                    explodicle@sh.itjust.worksE This user is from outside of this forum
                                    explodicle@sh.itjust.worksE This user is from outside of this forum
                                    [email protected]
                                    wrote on last edited by
                                    #121

                                    Because it only harms bots that ignore the "no crawl" directive, so your AI remains uncensored.

                                    M 1 Reply Last reply
                                    0
                                    • morrowind@lemmy.mlM [email protected]

                                      Thanks, I don't really care for generalizing instances and didn't really have a choice when I made my account.

                                      But also your definition is impossibly broad as you well and I'm pretty sure not the general consensus. The video doesn't define it as such either.

                                      For one thing, by your definition we can have absolutely no meaningful human relationships. I can explain this more later when I have time if you don't see what I mean

                                      ? Offline
                                      ? Offline
                                      Guest
                                      wrote on last edited by
                                      #122

                                      My explanation of authority wasn't meant as a definition but rather a brief summarisation of a complex concept, Andrew does a better job actually explaining it. Like pointing out that Authority is confused with a lot of different concepts like respect or influence. Which I'm starting to suspect is happening here. meaningful human relationships are based on mutual respect. This is not authority as it is voluntary, reciprocated and revoked as soon as the other party steps over the line. This is what I believe is the basis of society and what we need to return to in order to live a truly free life. In modern society in most interactions respect has been replaced with authority. People in positions of power even use them synonymously.

                                      1 Reply Last reply
                                      0
                                      • tea@programming.devT [email protected]
                                        This post did not contain any content.
                                        D This user is from outside of this forum
                                        D This user is from outside of this forum
                                        [email protected]
                                        wrote on last edited by
                                        #123

                                        Surprised at the level of negativity here. Having had my sites repeatedly DDOSed offline by Claudebot and others scraping the same damned thing over and over again, thousands of times a second, I welcome any measures to help.

                                        A dan@upvote.auD 2 Replies Last reply
                                        0
                                        • tea@programming.devT [email protected]
                                          This post did not contain any content.
                                          gmtom@lemmy.worldG This user is from outside of this forum
                                          gmtom@lemmy.worldG This user is from outside of this forum
                                          [email protected]
                                          wrote on last edited by
                                          #124

                                          "I used the AI to destroy the AI"

                                          C F 2 Replies Last reply
                                          0
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups