Cloudflare blocking Pale Moon and other browsers with smaller user bases
-
I'm actually surprised with the amount of hate towards Cloudflare in this thread. Should Cloudflare punish the vast majority of users by weakening their bot protection for the less than 1% of users who can just override their user agent?
Yes its should.
-
I'm actually surprised with the amount of hate towards Cloudflare in this thread. Should Cloudflare punish the vast majority of users by weakening their bot protection for the less than 1% of users who can just override their user agent?
If I remember correctly, Cloudflare openly defended hosting a well known Neo-Nazi forum.
-
Which bots use Palemoon as their UA string?
-
During my first job as a dev outta school, they had me writing scrapers. I was actually able to subvert it pretty easily using this package that doesn't appear to be maintained anymore https://github.com/VeNoMouS/cloudscraper
Was pretty surprised to learn that, at the time, they were only checking if JS was enabled, especially since CF is the gold standard for this sort of stuff. I'm sure this has changed?
Given that the last updates to this repo were five years ago, I'm not too sure if it's still valid. I don't follow Cloudflare bypasses but I am fairly certain there are more successful frameworks and services now. The landscape is evolving quickly. We are seeing a proliferation of "bot as a service", captcha passing farms, dedicated browsers for botting, newsletters, substacks, Discord servers, you name it. Then there are the methods you don't readily find much talk on like custom modified Chrome browsers. It's fascinating how much effort is being funneled into this field.
-
I get why you're frustrated and you have every right to be. I'm going to preface what I'm going to say next by saying I work in this industry. I'm not at Cloudflare but I am at a company that provides bot protection. I analyze and block bots for a living. Again, your frustrations are warranted.
-
Even if a site doesn't have sensitive information, it likely serves a captcha because of the amount of bots that do make requests that are scraping related. The volume of these requests can effectively DDoS them. If they're selling something, it can disrupt sales. So they lose money on sales and eat the load costs.
-
With more and more username and password leaks, credential stuffing is getting to be a bigger issue than anyone actually realizes. There aren't really good ways of pinpointing you vs someone that has somehow stolen your credentials. Bots are increasingly more and more sophisticated. Meaning, we see bots using aged sessions which is more in line with human behavior. Most of the companies implementing captcha on login segments do so to try and protect your data and financials.
-
The rise in unique, privacy based browsers is great and it's also hard to keep up with. It's been more than six months, but I've fingerprinted Pale Moon and, if I recall correctly, it has just enough red flags to be hard to discern between a human and a poorly configured bot.
Ok, enough apologetics. This is a cat and mouse game that the rest of us are being drug into. Sometimes I feel like this is a made up problem. Ultimately, I think this type of thing should be legislated. And before the bot bros jump in and say it's their right to scrape and take data it's not. Terms of use are plainly stated by these sites. They consider it stealing.
Thank you for coming to my Tedx Talk on bots.
You're definitely right that it's a game of one-upping each other. Unfortunately, it's now directed in a path that infringes on privacy of the users it aims to serve.
Since you're working in the internet security industry, what's your take on something like Altcha as opposed to more invasive means of protecting against both attacks?
-
-
Yes, the industry is well aware of this. We do behavioral detection on both sessions and IPs. This is fairly basic.
Yeah, it's fine as long as you don't block legitimate users. For example, when I use a VPN a lot of sites block me. Even when my actual IP is banned when I'm in China (4chan range bans Chinese IPs) or the website is blocked in China.
-
You're definitely right that it's a game of one-upping each other. Unfortunately, it's now directed in a path that infringes on privacy of the users it aims to serve.
Since you're working in the internet security industry, what's your take on something like Altcha as opposed to more invasive means of protecting against both attacks?
Trust me, my team and I often feel at odds with the part that infringes on privacy. As someone that enjoys and wants more privacy, I wish there were other solutions that didn't create a type of dragnet. If it assuages some of your fears, I've never heard of the fingerprinting being sold or used outside of detections.
ALTCHA uses a proof-of-work mechanism to protect your website, apps, APIs, and online services from spam and unwanted content.
Unlike other solutions, ALTCHA’s Captcha alternative is free, open-source and self-hosted, does not use cookies nor fingerprinting, does not track users.
Emphasis are mine. I honestly do not know how this statement is possible. Captcha-less, proof-of-work solutions have to fingerprint on some level. It's essentially having the browser prove it is what it claims to be. I get what they're trying to say but it's marketing. That said, I don't know everything and maybe they have some method I'm not aware of. Grains of salt all around.
-
This post did not contain any content.
I was planning on moving away from Cloudflare to European providers anyway, so this just adds fuel to the fire.
I'm considering using BunnyDNS for DNS management, not using a CDN at all, and using Scaleway for serverless functions.
-
Small companies use CF as well. It really is one of the best ways to prevent all sorts of bad actors
One of the easiest, perhaps. Not best. Anything that gives a single entity control over so much of the internet, and positions them to snoop on so much of everyone's communications, will never be "best".
-
Lol what?
I thought they just did rate limiting and such, I can't believe they do SSL as well.
Have you ever tried to visit a web site and found a Cloudflare error page instead? It might have looked like this:
Do you know how they're able to insert that error page into the response that reaches your browser, even though it's an https connection and your browser assures you that it's "secure"?
Clouldflare is able to do this because they are a middle-man between you and the site. They can eavesdrop and/or alter anything sent or received on that connection.
-
This post did not contain any content.
Need to start spoofing user agent strings again.
-
Trust me, my team and I often feel at odds with the part that infringes on privacy. As someone that enjoys and wants more privacy, I wish there were other solutions that didn't create a type of dragnet. If it assuages some of your fears, I've never heard of the fingerprinting being sold or used outside of detections.
ALTCHA uses a proof-of-work mechanism to protect your website, apps, APIs, and online services from spam and unwanted content.
Unlike other solutions, ALTCHA’s Captcha alternative is free, open-source and self-hosted, does not use cookies nor fingerprinting, does not track users.
Emphasis are mine. I honestly do not know how this statement is possible. Captcha-less, proof-of-work solutions have to fingerprint on some level. It's essentially having the browser prove it is what it claims to be. I get what they're trying to say but it's marketing. That said, I don't know everything and maybe they have some method I'm not aware of. Grains of salt all around.
I definitely understand. That's good to hear there hasn't been a direct pipeline to selling fingerprint data established yet.
Thanks for checking it out. Hopefully there is a best of both worlds in what they are advertising but I get that technology isn't magic either.
-
Given that the last updates to this repo were five years ago, I'm not too sure if it's still valid. I don't follow Cloudflare bypasses but I am fairly certain there are more successful frameworks and services now. The landscape is evolving quickly. We are seeing a proliferation of "bot as a service", captcha passing farms, dedicated browsers for botting, newsletters, substacks, Discord servers, you name it. Then there are the methods you don't readily find much talk on like custom modified Chrome browsers. It's fascinating how much effort is being funneled into this field.
Oh i can definitely see custom browsers being useful in that area. I remember the JavaScript navigator properties were always such a PITA as there was nothing you could really do to get around what they exposed
-
I get why you're frustrated and you have every right to be. I'm going to preface what I'm going to say next by saying I work in this industry. I'm not at Cloudflare but I am at a company that provides bot protection. I analyze and block bots for a living. Again, your frustrations are warranted.
-
Even if a site doesn't have sensitive information, it likely serves a captcha because of the amount of bots that do make requests that are scraping related. The volume of these requests can effectively DDoS them. If they're selling something, it can disrupt sales. So they lose money on sales and eat the load costs.
-
With more and more username and password leaks, credential stuffing is getting to be a bigger issue than anyone actually realizes. There aren't really good ways of pinpointing you vs someone that has somehow stolen your credentials. Bots are increasingly more and more sophisticated. Meaning, we see bots using aged sessions which is more in line with human behavior. Most of the companies implementing captcha on login segments do so to try and protect your data and financials.
-
The rise in unique, privacy based browsers is great and it's also hard to keep up with. It's been more than six months, but I've fingerprinted Pale Moon and, if I recall correctly, it has just enough red flags to be hard to discern between a human and a poorly configured bot.
Ok, enough apologetics. This is a cat and mouse game that the rest of us are being drug into. Sometimes I feel like this is a made up problem. Ultimately, I think this type of thing should be legislated. And before the bot bros jump in and say it's their right to scrape and take data it's not. Terms of use are plainly stated by these sites. They consider it stealing.
Thank you for coming to my Tedx Talk on bots.
Terms of use are plainly stated by these sites. They consider it stealing.
I consider it more trespassing than stealing myself.
-
-
Have you ever tried to visit a web site and found a Cloudflare error page instead? It might have looked like this:
Do you know how they're able to insert that error page into the response that reaches your browser, even though it's an https connection and your browser assures you that it's "secure"?
Clouldflare is able to do this because they are a middle-man between you and the site. They can eavesdrop and/or alter anything sent or received on that connection.
I thought that was for their hosted websites, had no idea whether they even do hosting/cloud infra.
But yes, I hate them to my core.
-
I was planning on moving away from Cloudflare to European providers anyway, so this just adds fuel to the fire.
I'm considering using BunnyDNS for DNS management, not using a CDN at all, and using Scaleway for serverless functions.
Maybe is against the ToS but I've used github as CDN for free in the past... Might work for you.
I never felt it was wrong, it was around the time of the Microsoft acquisition.
-
Oh i can definitely see custom browsers being useful in that area. I remember the JavaScript navigator properties were always such a PITA as there was nothing you could really do to get around what they exposed
There's a whole world of tools you can use that do that for you now. It's easier than ever. To me it's concerning. The level of automation, coupled with a halfway decent LLM, can give you the ability to summon hordes of fake humans to social media. I can't help but think it's why X and Reddit don't use any kind of anti-bot solution.
-
Welcome to bot detection. It's a cat and mouse game, an ever changing battle where each side makes moves and counter moves. You can see this with the creation of captcha-less challenges.
But to say captcha are useless because bots can pass them is somewhat similar to saying your antivirus is useless because certain malware and ransomware can bypass it.
Uh, no, they are better than humans at solving them.
-
Maybe is against the ToS but I've used github as CDN for free in the past... Might work for you.
I never felt it was wrong, it was around the time of the Microsoft acquisition.
I appreciate the suggestion, but Github is also an American company. I've been moving my git repositories to Codeberg.
My sites don't get enough traffic to warrant a CDN really, but if necessary, BunnyCDN looks like it can fit the bill. Plus, my static sites are in Scaleway object storage.
-
LibreWolf is next, and it's not exactly niche. In seeing it more and more, and LW defaults, even dropping resist settings, gets bounced by CloudFlare every time.
Wouldn't that also block Firefox by proxy?
