How to Set Up SSH Key Authentication on a Linux VPS: Complete Guide

Password-based SSH authentication is one of the most common entry points for attackers. Bots continuously scan the internet for open port 22 and attempt thousands of username/password combinations per second (brute-force attacks). The elegant, industry-recommended solution is SSH key authentication — a cryptographic mechanism that is both far more secure and, once configured, more convenient to use.

This guide walks you through the entire process — from generating a key pair to disabling password authentication — applicable to any Linux server, whether a high-performance NVMe VPS or a dedicated machine.

What Is SSH Key Authentication?

SSH (Secure Shell) supports two primary authentication methods:

Password — simple, but vulnerable to brute-force attacks.
Key pair — a private key (stays on your computer) and a public key (copied to the server). The server verifies your identity without transmitting any password over the network.

The private key can be additionally protected with a local passphrase, adding a second layer of security.

Step 1: Generate an SSH Key Pair on Your Local Machine

Open a terminal on your machine (Linux, macOS, or WSL on Windows) and run:

ssh-keygen -t ed25519 -C "[email protected]"

Why ed25519? It's the modern recommended algorithm — shorter keys, stronger security than RSA-2048 or RSA-4096. On very old servers, use ssh-keygen -t rsa -b 4096.

The command will ask for:
1. File location — press Enter to accept the default (~/.ssh/id_ed25519).
2. Passphrase — highly recommended; protects your private key if someone gains access to your computer.

Two files are created:
- ~/.ssh/id_ed25519 — the private key (never share this!)
- ~/.ssh/id_ed25519.pub — the public key (this gets copied to the server)

Step 2: Copy the Public Key to the Server

Method 1 — `ssh-copy-id` (recommended on Linux/macOS)

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@SERVER_IP

This command authenticates with your current password, automatically creates ~/.ssh/authorized_keys, and appends the key.

Method 2 — Manual (Windows or when `ssh-copy-id` is unavailable)

# Display the public key and copy it
cat ~/.ssh/id_ed25519.pub

On the server, as your user:

mkdir -p ~/.ssh
chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys
# Paste the public key content, save the file
chmod 600 ~/.ssh/authorized_keys

Step 3: Test Key-Based Authentication

Before disabling password login, open a new terminal window and verify the connection:

ssh -i ~/.ssh/id_ed25519 user@SERVER_IP

If you connected without a password (or using the key's passphrase), everything is working. Do not close your current session until confirmed!

If you're using a SSD VPS at CLIQHOST, find your server IP in the client control panel.

Step 4: Disable Password Authentication in sshd_config

This is the critical step. Edit the SSH configuration file:

sudo nano /etc/ssh/sshd_config

Find and modify (or add) the following directives:

PasswordAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no
ChallengeResponseAuthentication no

On Ubuntu 22.04+, also check files in /etc/ssh/sshd_config.d/ — they may override the main settings:

grep -r 'PasswordAuthentication' /etc/ssh/sshd_config.d/

After making changes, restart the SSH service:

sudo systemctl restart sshd

Step 5: Additional Security Settings in sshd_config

While you're in the file, apply these best practices:

# Disable direct root login
PermitRootLogin no

# Restrict which users can SSH
AllowUsers your_username

# Change the default port (optional, reduces log noise)
Port 2222

# Idle session timeout
ClientAliveInterval 300
ClientAliveCountMax 2

Warning if you change the port: update your firewall rules before restarting, or you'll lock yourself out!

# UFW
sudo ufw allow 2222/tcp

# iptables
sudo iptables -A INPUT -p tcp --dport 2222 -j ACCEPT

For comprehensive firewall and server security management, explore CLIQHOST's server management services.

Step 6: Managing Keys for Multiple Users or Servers

The `~/.ssh/config` File on Your Local Machine

If you manage multiple servers, create a config file to simplify connections:

# ~/.ssh/config
Host vps-prod
    HostName 185.x.x.x
    User adminuser
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Host vps-dev
    HostName 185.x.x.y
    User devuser
    Port 22
    IdentityFile ~/.ssh/id_ed25519_dev

Now you simply connect with:

ssh vps-prod

Key Rotation (Best Practice)

Generate a new key periodically (annually or when team members leave), add it to authorized_keys, then remove the old one. Never leave unused keys on the server.

Managing SSH Keys for Admin Teams

On a managed dedicated server where a team operates, each administrator should have their own key pair. Keys are added to authorized_keys one per line:

# authorized_keys — each line = one public key
ssh-ed25519 AAAA...key_admin1... [email protected]
ssh-ed25519 AAAA...key_admin2... [email protected]

When an admin leaves, simply delete their line — no need to reset the entire server's keys.

Troubleshooting Common Issues

"Permission denied (publickey)"

Check permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Ensure the public key is on a single line in authorized_keys with no line breaks.
Use verbose mode for diagnostics: ssh -vvv user@server

SELinux Blocking SSH (CentOS/RHEL/AlmaLinux)

restorecon -Rv ~/.ssh

Locked Out After Disabling Passwords

If you disabled password auth and the key isn't working, access the VPS VNC console in the CLIQHOST control panel, or reach out to our support team — access can be restored at the hypervisor level.

Using SSH Agent for Convenience

If you set a passphrase on your key (you should!), you don't need to type it every time. SSH Agent keeps it in memory for the duration of the session:

# Start the agent
eval "$(ssh-agent -s)"

# Add your key (enter passphrase once)
ssh-add ~/.ssh/id_ed25519

On macOS, save it to Keychain: ssh-add --apple-use-keychain ~/.ssh/id_ed25519

Final Checklist — SSH Key Authentication

[ ] Key pair generated with ed25519
[ ] Public key copied to ~/.ssh/authorized_keys on the server
[ ] Successful login tested in a new terminal window
[ ] PasswordAuthentication no set in sshd_config
[ ] PermitRootLogin no enabled
[ ] Firewall updated if port was changed
[ ] SSH service restarted
[ ] Old/unused keys removed

Conclusion

SSH key authentication is one of the most effective security measures you can apply to a Linux server. Setup takes 10–15 minutes and virtually eliminates the risk of brute-force attacks on SSH access.

If you're running a serious project on a high-performance NVMe VPS or a dedicated server, securing SSH access is the first step in a solid security strategy.

Not sure how to configure your server correctly? The CLIQHOST team offers complete Linux server management services — from initial hardening to continuous monitoring. Contact us and we'll set up your server according to security best practices.

How to Set Up SSH Key Authentication on a Linux VPS: Complete Guide