Cloudflare blocking Pale Moon and other browsers with smaller user bases
-
So cute
It's not much, but it's home.
-
Dude, thank you for this context. I was already aware of these considerations but just wanted to thank you for sharing this with everyone. Its participation like this that makes the internet a better place.
That's very kind of you. Thank you for the kind words.
-
Thank you for that info, very helpful.
Thank you for reading and considering the information.
-
I get why you're frustrated and you have every right to be. I'm going to preface what I'm going to say next by saying I work in this industry. I'm not at Cloudflare but I am at a company that provides bot protection. I analyze and block bots for a living. Again, your frustrations are warranted.
-
Even if a site doesn't have sensitive information, it likely serves a captcha because of the amount of bots that do make requests that are scraping related. The volume of these requests can effectively DDoS them. If they're selling something, it can disrupt sales. So they lose money on sales and eat the load costs.
-
With more and more username and password leaks, credential stuffing is getting to be a bigger issue than anyone actually realizes. There aren't really good ways of pinpointing you vs someone that has somehow stolen your credentials. Bots are increasingly more and more sophisticated. Meaning, we see bots using aged sessions which is more in line with human behavior. Most of the companies implementing captcha on login segments do so to try and protect your data and financials.
-
The rise in unique, privacy based browsers is great and it's also hard to keep up with. It's been more than six months, but I've fingerprinted Pale Moon and, if I recall correctly, it has just enough red flags to be hard to discern between a human and a poorly configured bot.
Ok, enough apologetics. This is a cat and mouse game that the rest of us are being drug into. Sometimes I feel like this is a made up problem. Ultimately, I think this type of thing should be legislated. And before the bot bros jump in and say it's their right to scrape and take data it's not. Terms of use are plainly stated by these sites. They consider it stealing.
Thank you for coming to my Tedx Talk on bots.
Ever heard of counting attempts? Log the IP, present a CAPTCHA after 100 requests in a minute.
Besides, if I wrote a bot I would run a browser dialer from Chrome. It would request your site in a Chrome tab and appear completely legitimate to your stupid fingerprinting scripts
-
-
Ever heard of counting attempts? Log the IP, present a CAPTCHA after 100 requests in a minute.
Besides, if I wrote a bot I would run a browser dialer from Chrome. It would request your site in a Chrome tab and appear completely legitimate to your stupid fingerprinting scripts
Yes, the industry is well aware of this. We do behavioral detection on both sessions and IPs. This is fairly basic.
-
Ever heard of counting attempts? Log the IP, present a CAPTCHA after 100 requests in a minute.
Besides, if I wrote a bot I would run a browser dialer from Chrome. It would request your site in a Chrome tab and appear completely legitimate to your stupid fingerprinting scripts
Ever heard of counting attempts? Log the IP, present a CAPTCHA after 100 requests in a minute.
Ever heard of IP rotation?
This is one malicious source rotating through IPs over the course of 24 hours. They're attempting to credential stuff my logins ( on a production service ).
-
This post did not contain any content.
So make useragent sniffing useless by all being Chrome?
-
I get why you're frustrated and you have every right to be. I'm going to preface what I'm going to say next by saying I work in this industry. I'm not at Cloudflare but I am at a company that provides bot protection. I analyze and block bots for a living. Again, your frustrations are warranted.
-
Even if a site doesn't have sensitive information, it likely serves a captcha because of the amount of bots that do make requests that are scraping related. The volume of these requests can effectively DDoS them. If they're selling something, it can disrupt sales. So they lose money on sales and eat the load costs.
-
With more and more username and password leaks, credential stuffing is getting to be a bigger issue than anyone actually realizes. There aren't really good ways of pinpointing you vs someone that has somehow stolen your credentials. Bots are increasingly more and more sophisticated. Meaning, we see bots using aged sessions which is more in line with human behavior. Most of the companies implementing captcha on login segments do so to try and protect your data and financials.
-
The rise in unique, privacy based browsers is great and it's also hard to keep up with. It's been more than six months, but I've fingerprinted Pale Moon and, if I recall correctly, it has just enough red flags to be hard to discern between a human and a poorly configured bot.
Ok, enough apologetics. This is a cat and mouse game that the rest of us are being drug into. Sometimes I feel like this is a made up problem. Ultimately, I think this type of thing should be legislated. And before the bot bros jump in and say it's their right to scrape and take data it's not. Terms of use are plainly stated by these sites. They consider it stealing.
Thank you for coming to my Tedx Talk on bots.
But captchas have now proven useless, since bots are better at solving them now than humans?
-
-
Its kind of funny but thats how user agents have been for a while. It's historically just been browsers pretending to be one another.
Yeah and that's why it's one of the basics of the basics you learn as a software developer that you shouldn't sniff the useragent, because it's unreliable and causes issues. Yet all big webpages (especially those pretending to be a software) do it, causing issues.
-
And when Cloudflare is the proxy for a web site, it's Cloudflare that provides the HTTPS connection, meaning that you don't actually have an encrypted channel directly to the site. Cloudflare is the man-in-the-middle eavesdropping on all of your communications with that site. Your bank transactions, your medical records, your personal messages, etc.
Interesting. I'm going to keep this in mind.
Weird how much of a monopoly cloudflare has on the internet. I guess it's going to start being an indicator for me for services that have becomes "too big for their britches."
-
Disgusting and unsurprising.
Most web admins do not care. I've lost count of how many sites make me jump through CAPTCHAS or outright block me in private browsing or on VPN. Most of these sites have no sensitive information, or already know exactly who I am because I am already authenticating with my username and password. It's not something the actual site admins even think about. They click the button, say "it works on my machine!" and will happily blame any user whose client is not dead-center average.
Enter username, but first pass this CAPTCHA.
Enter password, but first pass this second CAPTCHA.
Here's another CAPTCHA because lol why not?
Some sites even have their RSS feed behind Cloudflare. And guess what that means? It means you can't fucking load it in a typical RSS reader. Good job!
The web is broken. JavaScript was a mistake. Return to
monkegopher.Fuck Cloudflare.
Ever been down the gemini rabbit hole? It's not perfect, but quite interesting.
-
But captchas have now proven useless, since bots are better at solving them now than humans?
Welcome to bot detection. It's a cat and mouse game, an ever changing battle where each side makes moves and counter moves. You can see this with the creation of captcha-less challenges.
But to say captcha are useless because bots can pass them is somewhat similar to saying your antivirus is useless because certain malware and ransomware can bypass it.
-
This post did not contain any content.
It is obvious that Cloudflare is being influenced to enforce browser monopolies. Imagine if Cloudflare existed in 2003 and stopped non Internet Explorer browsers. If you use cloudflare to "protect" your site you are discriminating against browser choice and are as bad as Microsoft in 1998.
-
then I guess they don't value us as much as they should and need to be reminded of their place under the consumers boot.
I mean the criticism against Cloudflare is 100% valid. Having a single service be the single point of failure for half the web, not to mention that they can read the contents of every single request they proxy, is a terrible joke.
But the service they provide is real. A small business/service just doesn't have the capabilities to handle a DDoS attack. And every minute their site is down means lost customers/users.
-
I can't use my Browser without it being created by a tech giant, cant use my new computer without having my software uefi signed by Microsoft, AI will soon need me to have my GPU licensed and registered.
The world is heading to crap.
You can, it'll cost more and give you less, but you can.
That's the way this works.
-
I get why you're frustrated and you have every right to be. I'm going to preface what I'm going to say next by saying I work in this industry. I'm not at Cloudflare but I am at a company that provides bot protection. I analyze and block bots for a living. Again, your frustrations are warranted.
-
Even if a site doesn't have sensitive information, it likely serves a captcha because of the amount of bots that do make requests that are scraping related. The volume of these requests can effectively DDoS them. If they're selling something, it can disrupt sales. So they lose money on sales and eat the load costs.
-
With more and more username and password leaks, credential stuffing is getting to be a bigger issue than anyone actually realizes. There aren't really good ways of pinpointing you vs someone that has somehow stolen your credentials. Bots are increasingly more and more sophisticated. Meaning, we see bots using aged sessions which is more in line with human behavior. Most of the companies implementing captcha on login segments do so to try and protect your data and financials.
-
The rise in unique, privacy based browsers is great and it's also hard to keep up with. It's been more than six months, but I've fingerprinted Pale Moon and, if I recall correctly, it has just enough red flags to be hard to discern between a human and a poorly configured bot.
Ok, enough apologetics. This is a cat and mouse game that the rest of us are being drug into. Sometimes I feel like this is a made up problem. Ultimately, I think this type of thing should be legislated. And before the bot bros jump in and say it's their right to scrape and take data it's not. Terms of use are plainly stated by these sites. They consider it stealing.
Thank you for coming to my Tedx Talk on bots.
Thanks for sharing!
-
-
Interesting. I'm going to keep this in mind.
Weird how much of a monopoly cloudflare has on the internet. I guess it's going to start being an indicator for me for services that have becomes "too big for their britches."
Small companies use CF as well. It really is one of the best ways to prevent all sorts of bad actors
-
Thanks for sharing!
Thanks for reading and commenting!
-
This post did not contain any content.
What doesn't work with Lynx is a wrong website.
-
Thanks for reading and commenting!
During my first job as a dev outta school, they had me writing scrapers. I was actually able to subvert it pretty easily using this package that doesn't appear to be maintained anymore https://github.com/VeNoMouS/cloudscraper
Was pretty surprised to learn that, at the time, they were only checking if JS was enabled, especially since CF is the gold standard for this sort of stuff. I'm sure this has changed?
