How to secure a new Linux server

A default Linux install is not insecure, but it is not configured for the fact that it is about to sit on the public internet with a routable IP address either. Within hours of a new VPS going live, automated scanners will find it and start trying default usernames and passwords against SSH — that is simply background noise on the internet, not a targeted attack, and this checklist is what stops it from being a problem.

Do these in order. Several of them can lock you out if done in the wrong sequence or without checking each step first.

1. Update everything first

sudo apt update && sudo apt upgrade -y      # Ubuntu / Debian
sudo dnf upgrade -y                         # RHEL / Rocky / AlmaLinux

A server that has not been updated since its image was built is the single most common way a VPS ends up compromised — not through anything you did, but through a vulnerability that was already patched upstream before you ever logged in. Keeping this current on an ongoing basis is covered in how to keep a Linux server patched.

2. Create a personal sudo account

Stop logging in as root for routine work. A named account with sudo rights gives you an audit trail, and it means a compromised credential is one account rather than the account with no restrictions on it whatsoever.

sudo adduser yourname
sudo usermod -aG sudo yourname      # Ubuntu / Debian — group is "sudo"
sudo usermod -aG wheel yourname     # RHEL / Rocky / AlmaLinux — group is "wheel"

Full walkthrough in how to create a non-root sudo user.

3. Switch SSH to key-based login

Generate a key pair on your own machine if you have not already:

ssh-keygen -t ed25519 -C "you@example.com"

Copy the public key to your new sudo account:

ssh-copy-id yourname@203.0.113.10

Open a new terminal window and confirm you can log in as that user with no password prompt before changing anything else. Only then edit /etc/ssh/sshd_config:

PasswordAuthentication no
PermitRootLogin no
Protocol 2

sudo systemctl restart ssh     # Debian / Ubuntu
sudo systemctl restart sshd    # RHEL / Rocky / AlmaLinux

4. Enable a firewall, allowing SSH first

sudo ufw allow OpenSSH
sudo ufw enable

On RHEL-family systems, firewalld is usually already running by default — check with sudo firewall-cmd --state before assuming you need to enable it. Either way, allow SSH explicitly before adding or removing anything else. Full detail in setting up a firewall on your VPS.

5. Install Fail2ban

Fail2ban watches log files for repeated failed login attempts and temporarily blocks the offending IP address at the firewall. It does not replace key-based login — it reduces the noise and the small residual risk from any service that still accepts a password, and it is worth having even with password login already disabled, since it also covers other services you might expose later.

sudo apt install fail2ban -y      # Ubuntu / Debian
sudo dnf install fail2ban -y      # RHEL / Rocky / AlmaLinux

Configuration specifics — jails, ban times, and how to unblock yourself if you get banned by mistake — are covered in how to set up Fail2ban.

6. Consider changing the SSH port

Moving SSH off port 22 does not make the server more secure against a determined attacker — it only reduces the volume of automated background scanning that reaches your login prompt at all, which in turn makes your logs easier to read. It is a convenience measure, not a security one, and it is entirely optional. If you do it, update both sshd_config and your firewall rule together, and again confirm the new port works in a second session before closing the old one.

7. Turn on unattended security updates

sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure --priority=low unattended-upgrades

On RHEL-family systems, the equivalent is dnf-automatic:

sudo dnf install dnf-automatic -y
sudo systemctl enable --now dnf-automatic.timer

This closes the single largest gap between a server being secure on day one and quietly drifting out of date over the following months.

8. Keep an eye on it, going forward

Securing a server once is not the same as keeping it secure. Build a habit — weekly is reasonable for a small server — of checking:

• sudo fail2ban-client status sshd — anything being blocked repeatedly that you should know about.
• last -a — a list of recent logins, worth a glance for anything unfamiliar.
• df -h and free -h — resource exhaustion causes more outages than attacks do. See monitoring server resources.

None of this is a one-off task with a finish line. It is a routine, and the servers that stay secure are the ones where the routine actually happens.