How to harden against SSH brute-forcing?
-
A lot of things are "fine", but the cost of adding Netbird to your VPS is effectively zero, whether counted in dollars, time, or convenience.
Given the massive security benefits of using a VPN, and the lack of any meaningful downside to doing so, you'd be an idiot not to.
This is moving the goal posts. You went from "ssh is not fine to expose" to "VPN's add security". While the second is true, it's not what was being argued.
Never expose your SSH port on the public web,
Linux was designed as a multi user system. My college, Cal State Northridge, has an ssh server you can connect to, and put your site up. Many colleges continue to have a similar setup, and by putting stuff in your homedir you can have a website at no cost.
There are plenty of usecases which involve exposing ssh to the public internet.
And when it comes to raw vulnerabilities, ssh has had vastly less than stuff like apache httpd, which powers wordpress sites everywhere but has had so many path traversal and RCE vulns over the years.
-
Don't reject connections to port 22, honeypot it and ban on connection attempt.
honeypot
That's a lot more work.
-
Recently, I discovered that SSH of my VPS server is constantly battered as follows.
Apr 06 11:15:14 abastro-personal-arm sshd[102702]: Unable to negotiate with 218.92.0.201 port 53768: no matching key exchange method found. Their offer: diffie> Apr 06 11:30:29 abastro-personal-arm sshd[102786]: Unable to negotiate with 218.92.0.207 port 18464: no matching key exchange method found. Their offer: diffie> Apr 06 11:45:36 abastro-personal-arm sshd[102881]: Unable to negotiate with 218.92.0.209 port 59634: no matching key exchange method found. Their offer: diffie> Apr 06 12:01:02 abastro-personal-arm sshd[103019]: Unable to negotiate with 218.92.0.203 port 16976: no matching key exchange method found. Their offer: diffie> Apr 06 12:05:49 abastro-personal-arm sshd[103066]: Unable to negotiate with 218.92.0.212 port 49130: no matching key exchange method found. Their offer: diffie> Apr 06 12:07:09 abastro-personal-arm sshd[103077]: Connection closed by 162.142.125.122 port 56110 [preauth] Apr 06 12:12:18 abastro-personal-arm sshd[103154]: Connection closed by 45.79.181.223 port 22064 [preauth] Apr 06 12:12:19 abastro-personal-arm sshd[103156]: Connection closed by 45.79.181.223 port 22078 [preauth] Apr 06 12:12:20 abastro-personal-arm sshd[103158]: Connection closed by 45.79.181.223 port 22112 [preauth] Apr 06 12:21:26 abastro-personal-arm sshd[103253]: Connection closed by 118.25.174.89 port 36334 [preauth] Apr 06 12:23:39 abastro-personal-arm sshd[103282]: Unable to negotiate with 218.92.0.252 port 59622: no matching key exchange method found. Their offer: diffie> Apr 06 12:26:38 abastro-personal-arm sshd[103312]: Connection closed by 92.118.39.73 port 44400 Apr 06 12:32:22 abastro-personal-arm sshd[103373]: Unable to negotiate with 218.92.0.203 port 57092: no matching key exchange method found. Their offer: diffie> Apr 06 12:49:48 abastro-personal-arm sshd[103556]: error: maximum authentication attempts exceeded for root from 98.22.89.155 port 53675 ssh2 [preauth] Apr 06 12:49:48 abastro-personal-arm sshd[103556]: Disconnecting authenticating user root 98.22.89.155 port 53675: Too many authentication failures [preauth] Apr 06 12:49:51 abastro-personal-arm sshd[103558]: error: maximum authentication attempts exceeded for root from 98.22.89.155 port 53775 ssh2 [preauth] Apr 06 12:49:51 abastro-personal-arm sshd[103558]: Disconnecting authenticating user root 98.22.89.155 port 53775: Too many authentication failures [preauth] Apr 06 12:49:53 abastro-personal-arm sshd[103561]: error: maximum authentication attempts exceeded for root from 98.22.89.155 port 53829 ssh2 [preauth] Apr 06 12:49:53 abastro-personal-arm sshd[103561]: Disconnecting authenticating user root 98.22.89.155 port 53829: Too many authentication failures [preauth] Apr 06 12:49:54 abastro-personal-arm sshd[103563]: Connection closed by 98.22.89.155 port 53862 [preauth] Apr 06 12:50:41 abastro-personal-arm sshd[103576]: Invalid user from 75.12.134.50 port 36312 Apr 06 12:54:26 abastro-personal-arm sshd[103621]: Connection closed by 165.140.237.71 port 54236 Apr 06 13:01:26 abastro-personal-arm sshd[103702]: Connection closed by 193.32.162.132 port 33380 Apr 06 13:03:40 abastro-personal-arm sshd[103724]: Unable to negotiate with 218.92.0.204 port 60446: no matching key exchange method found. Their offer: diffie> Apr 06 13:11:49 abastro-personal-arm sshd[103815]: Received disconnect from 165.140.237.71 port 50952:11: [preauth] Apr 06 13:11:49 abastro-personal-arm sshd[103815]: Disconnected from authenticating user root 165.140.237.71 port 50952 [preauth] Apr 06 13:19:08 abastro-personal-arm sshd[103897]: Unable to negotiate with 218.92.0.208 port 59274: no matching key exchange method found. Their offer: diffie> Apr 06 13:33:36 abastro-personal-arm sshd[104066]: Received disconnect from 165.140.237.71 port 50738:11: [preauth] Apr 06 13:33:36 abastro-personal-arm sshd[104066]: Disconnected from authenticating user ubuntu 165.140.237.71 port 50738 [preauth] Apr 06 13:34:50 abastro-personal-arm sshd[104079]: Unable to negotiate with 218.92.0.204 port 44816: no matching key exchange method found. Their offer: diffie> Apr 06 13:50:32 abastro-personal-arm sshd[104249]: Unable to negotiate with 218.92.0.206 port 27286: no matching key exchange method found. Their offer: diffie> Apr 06 13:51:58 abastro-personal-arm sshd[104261]: Received disconnect from 165.140.237.71 port 50528:11: [preauth] Apr 06 13:51:58 abastro-personal-arm sshd[104261]: Disconnected from authenticating user root 165.140.237.71 port 50528 [preauth] Apr 06 14:01:25 abastro-personal-arm sshd[104351]: Invalid user from 65.49.1.29 port 18519 Apr 06 14:01:28 abastro-personal-arm sshd[104351]: Connection closed by invalid user 65.49.1.29 port 18519 [preauth]
As you can see, it is happening quite frequently, and I am worried one might break in at some point. Since SSH access guards users with root-access, it can be quite serious once penetrated.
How do I harden against these kind of attacks? Because this is VPS, disabling SSH is a no-go (SSH is my only entry of access). Are there ways to stop some of these attackers?As always, thanks in advance!
- harden sshd
- use fail2ban or even better CrowdStrike
- use a tool like the following to have a next-gen security solution: https://github.com/mrash/fwknop
-
You don't. This is normal. Ensure key-only auth, ensure you do not login directly as root, maybe install fail2ban and you're good. Some people move the port to a nonstandard one, but that only helps with automated scanners not determined attackers.
You could look into port-knocking if you want it really safe.
Some people move the port to a nonstandard one, but that only helps with automated scanners not determined attackers.
While true, cleaning up your logs such that you can actually see a determined attacker rather than it just getting buried in the noise is still worthwhile.
-
A few replies here give the correct advice. Others are just way off.
To those of you who wrote anything else than "disable passwords, use key based login only and you're good" - please spend more time learning the subject before offering up advice to others.
(fail2ban is nice to run in addition, I do so myself, but it's more for to stop wasting resources than having to do with security since no one is bruteforcing keys)
There's more to it than that.
I recommend geoip blocking anything outside of your expected operating regions in addition to using key-based logins.
iptables
operates at a lower level in the network stack than SSH, so the vulnerability surface is a lot lower, and blocking before something actually looks at the packets cleans up the logs. This is huge because it makes it a lot more obvious when there's a legitimate attack.Cover yourself with layers:
- block obviously bad packets at the firewall level
- eliminate insecure modes of login (only allow key-based login)
- something like fail2ban to ban the few who make it through 1 & 2
- use a secure root password so if someone does get in, they're less likely to get root access
- have your services run as non-privileged users to limit issues if something gets compromised
If you only do one thing, it should be only allowing key-based logins. If you do two, run SSH on a non-standard port or set up geoip blocking (second is more work, but a lot more effective).
-
In addition to other advice you could also use SSH over Wireguard. Wireguard basically makes the open port invisible. If you don't provide the proper key upfront you get no response. To an attacker the port might as well be closed.
Here's at least one article on the subject: https://rair.dev/wireguard-ssh/
Exactly. No root login and no password login are always useful as basic measures, but after that Wireguard is perfect tool for this, no weird rituals required and also quite useful for any other services you don't want and/or need to expose to the internet as well.
-
And why exactly is that more secure?
Main reason is that if you don't already have the right key, VPN doesn't even respond, it's just a black hole where all packets get dropped. SSH on the other hand will respond whether or not you have a password or a key, which lets the attacker know that there's something there listening.
That's not to say SSH is insecure, I think it's fine to expose once you take some basic steps to lock it down, just answering the question.
-
And why exactly is that more secure?
I don't know about Netbird specifically, but adding a VPN does a few things:
- a port scan of your VPS/router won't show an SSH or VPN port active - Wireguard only acknowledges packets if your key is valid (massively more useful than just changing the port)
- compromising both a VPN and SSH is difficult, you'd have to chain exploits together
- if your VPN is hosted by a separate service (e.g. something like Tailscale), it will be very unlikely to share vulnerabilities with your hosted SSH server
-
I don’t really get the love for fail2ban. Sure, it helps keep your logs clean, but with a solid SSH setup (root disabled, SSH keys enforced), I’m not bothered by the login attempts.
You should be. Most of it's noise, but if there's a serious attack, you'll appreciate clean logs.
I think fail2ban is nice as like a third or fourth layer of defense. In order of my priorities:
- key-only login, root login completely disabled
- solid root password, and user privilege separation (have each service use its own user)
- geoip bans - if you never plan to support clients in a given region, block it at the firewall level (or better yet, whitelist the handful of regions you care about); I do this by port, so SSH gets a much more restricted set of allowed regions than HTTPS
- fail2ban - especially if you have a relatively large whitelist
- only access SSH over a Wireguard VPN - Wireguard doesn't show up in port scans, and SSH can bind to the VPN host instead of 0.0.0.0, so the ability to login via SSH will be completely hidden
If you're not going to do 3-5, at least change the default SSH port to cut down on log noise.
-
Move the ssh port to higher ranges, 30-60000. That alone will stop 99% of the attacks
Disable root logins, now usernames must be guessed too which will make success even lower
Then require SSH keys
At that point it's like being in a nuclear fallout nshelter behind a 3 meter thick steel door and you can hear some zombies scratching on the outside... I'm not worried about any of that shit
For added funs run an SSH tarpit to fuck with the attackers, something like endlessh.
-
This is moving the goal posts. You went from "ssh is not fine to expose" to "VPN's add security". While the second is true, it's not what was being argued.
Never expose your SSH port on the public web,
Linux was designed as a multi user system. My college, Cal State Northridge, has an ssh server you can connect to, and put your site up. Many colleges continue to have a similar setup, and by putting stuff in your homedir you can have a website at no cost.
There are plenty of usecases which involve exposing ssh to the public internet.
And when it comes to raw vulnerabilities, ssh has had vastly less than stuff like apache httpd, which powers wordpress sites everywhere but has had so many path traversal and RCE vulns over the years.
We're in selfhosted. If you have to bring up use cases that are in no way relevant to 99% of self hosters to justify your argument, you don't have an argument.
-
Eh, while I agree that some recommendations are dodgy at best, I'll argue that Wireguard is not only adding to security, it also makes Fail2Ban obsolete. Due to the way it works, you'll completely hide the fact that you're even running a SSH server at all, and this includes even Wireguard itself. More importantly though, it's pretty much impossible to set up Wireguard in an insecure way, whereas SSH provides you with plenty of footguns. You're not risking locking yourself out either.
Also, security comes in layers.
-
Eh, while I agree that some recommendations are dodgy at best, I'll argue that Wireguard is not only adding to security, it also makes Fail2Ban obsolete. Due to the way it works, you'll completely hide the fact that you're even running a SSH server at all, and this includes even Wireguard itself. More importantly though, it's pretty much impossible to set up Wireguard in an insecure way, whereas SSH provides you with plenty of footguns. You're not risking locking yourself out either.
Also, security comes in layers.
You’re not risking locking yourself out either.
In a VPS, you should always be able to fall back to the web console. So locking yourself out shouldn't be a major concern.
-
For added funs run an SSH tarpit to fuck with the attackers, something like endlessh.
Well yeah, sure, but that doesn't really add to your security and it only costs you work and resources
-
One of the simplest is geoip blocks. Here's an article using iptables, and there may be a nicer way w/ whatever firewall you're using.
For reference, here are the areas I see in your logs (using this service
- 218.92.0.201 - China
- 162.142.125.122 - US (Michigan)
- 45.79.181.223 - US (New Jersey)
- 118.25.174.89 - China
- 92.118.39.73 - Romania
- 98.22.89.155 - US (Nebraska)
- 75.12.134.50 - US (Tennessee)
- 165.140.237.71 - US (Washington)
- 65.49.1.29 - US (California)
If you don't expect valid users to come from those areas, block them. A lot of those in the US are probably from VPN users, so be careful if people are using a VPN to connect to your services.
If you can do it w/ iptables, it'll be a lot more efficient than doing it at the application layer. I also recommend using something like fail2ban to block individual IPs within regions you care about to get any stragglers that make it through the first tier of blocks.
Thanks a lot! Geoblocking makes a lot of sense, will try!
-
Thanks a lot! Geoblocking makes a lot of sense, will try!
I highly recommend using key-based SSH authentication exclusively for all users on your server, and disallow root login as well.
Geoblocking mostly cuts down on the spam, but also constrains where an actual attack can come from. If there's some kind of zero-day attack on SSH, this will dramatically reduce the risk you're hit.
-
We're in selfhosted. If you have to bring up use cases that are in no way relevant to 99% of self hosters to justify your argument, you don't have an argument.
Again, this is distracting from the original argument to make some kind of tertiary argument unrelated to the original one: Is ssh secure to expose to the internet?
You said no. And that statement is inorrect.
-
We can’t ever stop this kind of stuff, but with something like fail2ban you can set it up to block on too many failures.
Really though - ensuring your system is kept up to date and uses strong passwords or use a SSH keys is the best defence. Blocking doesn’t prevent them from trying a few times. Moving SSH to a non standard port will stop most of the automated attacks but it won’t stop someone who is dedicated.
Thanks, I will try fail2ban. I am using ED25519 for ssh keys, it seems like it's the best defense on the ssh side.
Do you happen to know why this kind of attack is so prevalent? -
I highly recommend using key-based SSH authentication exclusively for all users on your server, and disallow root login as well.
Geoblocking mostly cuts down on the spam, but also constrains where an actual attack can come from. If there's some kind of zero-day attack on SSH, this will dramatically reduce the risk you're hit.
Fortunately my VPS (oracle) has set SSH authentication to be default. Disallowing root login sounds good, gotta try that as well.
-
Well yeah, sure, but that doesn't really add to your security and it only costs you work and resources
100% agree, that is a "totally for fun" exercise