Oli Warner About Contact Oli on Twitter Subscribe

Putting the Secure in SSH; hardening your shell server against most attacks

Tuesday, 4 November 2014 security ubuntu

SSH is the de facto remote access technique for Ubuntu and Linux servers and yet some of the defaults you’ll get from sudo apt-get install ssh can be downright dangerous in the wrong circumstances. This article will steer you around the biggest pitfalls to keep your server’s front door well protected.

It’s not enough to pick a strong password or hide your SSH server on a random port, good security is all about layers.

Ubuntu gives you fairly sane application defaults but in terms of actual security, it’s only going to stop you setting a <6 character password. Installing SSH on a brand new Ubuntu install with a weak password can result in a near-instant hack. I’m suggesting the following in order of importance. They can be applied independently but I would strongly suggest using all of them. There are some extension ideas at the end for the really paranoid.

And none of these is a substitute for other good-security practices. Keep your packages and websites updated, don’t run things you don’t need and use a firewall so only the right services are exposed to the internet.

Stop logging in as root directly

Generally speaking you shouldn’t need to run as root to get things done. Modern systems have utilities like sudo that make group-based permissions easy to allocate. I’m not going to enumerate all the reasons for not using root directly here but for the purposes of attacking a server, if root login is allowed, the attacker knows the username it should be attacking. If you disable it, they would also need to guess your username. Layers, baby.

So if you’re still running everything as root, let’s change that. Log into the server (as root) and we’ll start by creating a user in the sudo group. If you’re using an Ubuntu machine from before 12.04 —or one updated from then— you may need the admin group.

adduser <username> sudo

Try logging in as that user and make sure commands like sudo whoami work (it uses the account’s password, not root’s). Then we need to disable root login. This is a two-pronged attack. We’ll start by removing the root password:

sudo passwd -l root

Then we’ll explicitly disable root access via SSH by running:

sudo sed -ir 's/^(PermitRootLogin) .+/\1 no/' /etc/ssh/sshd_config
sudo /etc/init.d/ssh restart

Use key-based authentication and disable password authentication

A strong passphrase might keep you secure enough today but the botnets of tomorrow are going to chow through 20+ char phrases. What is impossible today will be childsplay in ten years time.

Key-based authentication uses algorithmic signing to verify a challenge/response and encrypt traffic between server and host. You keep a private half of the key in your $HOME/.ssh/ directory and upload a public hash to the server (under the remote users’ $HOME/.ssh/authorized_keys file).

While theoretically still crackable, it makes your “password” thousands of characters long and full of punctuation; essentially impossible to crack within the lifetime of any would-be attacker, even factoring in the advance of computing.

There are multiple types of key cryptography on offer. DSA and RSA are probably strong enough and are supported everywhere. Newer versions of Ubuntu support elliptical-curve algorithms. These are shorter and generally considered stronger encryption than a similar RSA or DSA keypair counterpart (please feel free to correct me in the comments).

Anyway, let’s create a key on the client. I’m using the fancy newer ed25519 crypto (suitable for 14.04 but not 12.04 clients and servers). If you need 12.04 support, use -t rsa.

ssh-keygen -t ed25519

This will give you an opportunity to add a password to your key. This is optional but it does make it two-factor authentication (something you have, something you know). Certainly worth considering if you feel there’s any risk of your private key falling into the wrong hands before you can revoke it.

When you’re done there, let’s upload the public half:

ssh-copy-id user@my.host

ssh-copy-id supports a -p <port> argument if your sshd already lives on a port other than 22.

OSX users can use a bit of simple scripting to emulate ssh-copy-id:

cat ~/.ssh/id_ed25519.pub | \
  ssh user@my.host "mkdir -p ~/.ssh; cat >> ~/.ssh/authorized_keys"

It’s a bit harder in Windows. You’re probably limited to RSA/DSA. The procedure is essentially:

 ssh-keygen -i -f id.ppk >> ~/.ssh/authorized_keys

You should be able to log in without a password now with just ssh user@my.host. Once all your SSH users have keys, and you have tested them, we can disable password login. Run sudoedit /etc/ssh/sshd_config and change

sudo sed -ir 's/^#?(PasswordAuthentication) .+/\1 no/' /etc/ssh/sshd_config
sudo /etc/init.d/ssh restart

Before you close you current connection, test the following in a new terminal:

ssh user@my.host                             # should work still, and
ssh -o PubkeyAuthentication=no user@my.host  # should deny entry.

Block brute-force attackers with fail2ban

Even with an “uncrackable” authentication method, we shouldn’t be over-confident. Why would we want to let people keep attacking our a once we know they’re trying to maliciously attack it? fail2ban is a great system that watches various log files for bad login attempts (amongst other things). If a number of failed attempts come from one IP within a certain timeframe, that IP is blocked for five minutes.

This level of time-ban isn’t going to ruin your life if you accidentally need to try five or six attempts at legitimately remembering your password but it’s going to completely arrest a true brute force attempt by limiting it to 72 attempts an hour. Given they would likely need to attempt billion of billions to crack your key-based auth, attackers will very quickly give up when faced with that throughput.

Installation is dead simple too:

sudo apt-get install fail2ban

It ships with a number of filters. /etc/fail2ban/filters.d/ contains the configurations for discovering perpetrators and /etc/fail2ban/jail.{conf,local} handle how they’re dealt with. There are SSH rules included by default.

Move the SSH server to another port

Every bot scanning for exploitable and crackable SSH servers is looking on port 22. Let’s move your SSH server off to a port other than 22.

Some people claim this is mere “security through obscurity” but they’re clearly not paying attention. Security through obscurity refers to situations when secrecy is your only line of defence. Having a hidden but otherwise open door, for example. We still have all our other security. We’re just adding a layer of camouflage.

This won’t stop a directed attack. Somebody checking all your ports will find it but that does take extra time and that means it stops the everyday people scanning for exploits.

The first thing to do is pick a port. You have a huge choice but I would suggest something in the 10,000-64,000 range that isn’t already taken and most importantly, something that isn’t in use by your server currently. Once you have that, log into the server and run:

sudo sed -ir 's/^(Port) .+/\1 12345/' /etc/ssh/sshd_config
sudo /etc/init.d/ssh restart

Obviously change the port number to the one you want. Finish by restarting SSH: It is critical that you test this on the new port before you disconnect your current session. Open a new session in a new client terminal:

ssh -p 12345 user@my.host

If that doesn’t work, something’s sqwonk.
Check /etc/ssh/sshd_config for obvious issues, fix them and try again.

You can make things easier by editing your local ~/.ssh/config to tell it about your new server configuration:

Host my.host
    User user
    Port 12345

Then you’ll be able to run ssh my.host and you’ll be straight in.

Network isolation

So a bunch of computers in China and Russia are still trying to crack your SSH server? Well, why do they have access to it in the first place? Most people only access their server from a limited number of networks. You have a couple of choices here.

You can whitelist a few addresses if you have static outgoing addresses. This is obviously unsuitable for people who are on dynamically assigned IPs (unless you know every IP your ISP can allocate you) but works well if you’re on a static IP or can route your connection through one. To enable this we run sudoedit /etc/ssh/sshd_config and add a line like:

AllowUsers *@1.2.3.* *@7.8.9.*

This is the equivalent of whitelisting any user from the 1.2.3.0/24 and 7.8.9.0/24 subnets. You can add wildcards further up the IP to widen the net. It wouldn’t take too long to work out your ISP’s entire range and enter that in. Similarly you could just add a single IP if you want to limit it to a single IP. Once you’ve done that, sudo /etc/init.d/ssh restart and test that it works before you disconnect.

If that’s impractical (because you hop between too many networks, you could adopt a GeoIP blocking in IPTables.

And if you have more than one server, you could designate one as the main SSH server and limit SSH connections to the others from that server’s IP. It complicates some connections (like deep SFTP access) but it ultimately protects the herd.

Ideas for going further

By this point we’re blocking all but the most concentrated attacks. Somebody is going to need a colossal botnet and more than one lifetime to crack your system through conventional means. There is still the risk that somebody could discover your SSH port and deploy an exploit against an unpatched vulnerability.

I can offer you here are a few more ideas for further obfuscating access to the SSH server:

Whichever combination of solutions you end up going with, remember that SSH isn’t the only way into a system. Make sure you use a firewall to block access to services that don’t need to be externally (ufw is a good one for Ubuntu users, just make sure you allow your new SSH port before you activate it) and keep things (including popular web-scripts like Wordpress and Drupal) up to date.

And if none of this meant anything to you, perhaps you shouldn’t be running a web-accessible server. I don’t mean that in an unkind way but just because anybody can afford to rent a VPS these days, doesn’t mean they’re ready for the job of looking after it. Keeping on top of security can be miserable at times. If you think you fall into that bracket, by all means learn on Ubuntu at home but go with managed web services. Or hire me or some other chump to look after your server for you.