SSH, or Secure Shell, is an encrypted protocol used to administer and communicate with servers. When working with a Linux server, you may often spend much of your time in a terminal session connected to your server through SSH.

While there are a few different ways to log in to an SSH server, in this guide, we’ll focus on setting up SSH keys. SSH keys provide an extremely secure way to log in to your server. For this reason, this is the method we recommend for all users.

Simplify deploying applications to servers with DigitalOcean App Platform. Deploy directly from GitHub in minutes.

An SSH server can authenticate clients using a variety of different methods. The most basic of these is password authentication, which is easy to use, but not the most secure.

Although passwords are sent to the server in a secure manner, they are generally not complex or long enough to be resistant to repeated, persistent attackers. Modern processing power combined with automated scripts make brute-forcing a password-protected account very possible. Although there are other methods of adding additional security (fail2ban, etc.), SSH keys prove to be a reliable and secure alternative.

SSH key pairs are two cryptographically secure keys that can be used to authenticate a client to an SSH server. Each key pair consists of a public key and a private key.

The private key is retained by the client and should be kept absolutely secret. Any compromise of the private key will allow the attacker to log in to servers that are configured with the associated public key without additional authentication. As an additional precaution, the key can be encrypted on disk with a passphrase.

The public key can be shared freely without any negative consequences. During authentication, SSH uses the key pair to prove that the client possesses the private key (without sending the private key over the network). In practice, the client signs data with the private key and the server verifies that signature using the public key.

The public key is uploaded to a remote server that you want to be able to log in to with SSH. The key is added to a special file in the user account you will use to log in called ~/.ssh/authorized_keys.

When a client attempts to authenticate using SSH keys, the server can test the client on whether they are in possession of the private key. If the client can prove that it owns the private key, a shell session is spawned or the requested command is executed.

The first step to configure SSH key authentication to your server is to generate an SSH key pair on your local computer.

To do this, we can use a special utility called ssh-keygen, which is included with the standard OpenSSH suite of tools. By default, this will create a 3072-bit RSA key pair.

On your local computer, generate an SSH key pair by typing:

ssh-keygen

Output
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):

The utility will prompt you to select a location for the keys that will be generated. By default, the keys will be stored in the ~/.ssh directory within your user’s home directory. The private key will be called id_rsa and the associated public key will be called id_rsa.pub.

Usually, it is best to stick with the default location at this stage. Doing so will allow your SSH client to automatically find your SSH keys when attempting to authenticate. If you would like to choose a non-standard path, type that in now, otherwise, press ENTER to accept the default.

If you had previously generated an SSH key pair, you may see a prompt that looks like this:

Output
/home/username/.ssh/id_rsa already exists.
Overwrite (y/n)?

If you choose to overwrite the key on disk, you will not be able to authenticate using the previous key anymore. Be very careful when selecting yes, as this is a destructive process that cannot be reversed.

Output
Created directory '/home/username/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:

Next, you will be prompted to enter a passphrase for the key. This is an optional passphrase that can be used to encrypt the private key file on disk.

You may be wondering what advantages an SSH key provides if you still need to enter a passphrase. Some of the advantages are:

• The private SSH key (the part that can be passphrase protected), is never exposed on the network. The passphrase is only used to decrypt the key on the local machine. This means that network-based brute forcing will not be possible against the passphrase.
• The private key is kept within a restricted directory. The SSH client will not recognize private keys that are not kept in restricted directories. The key itself must also have restricted permissions (read and write only available for the owner). This means that other users on the system cannot snoop.
• Any attacker hoping to crack the private SSH key passphrase must already have access to the system. This means that they will already have access to your user account or the root account. If you are in this position, the passphrase can prevent the attacker from immediately logging in to your other servers. This will hopefully give you time to create and implement a new SSH key pair and remove access from the compromised key.

Since the private key is never exposed to the network and is protected through file permissions, this file should never be accessible to anyone other than you (and the root user). The passphrase serves as an additional layer of protection in case these conditions are compromised.

A passphrase is an optional addition. If you enter one, you will have to provide it every time you use this key (unless you are running SSH agent software that stores the decrypted key). We recommend using a passphrase, but if you do not want to set a passphrase, you can press ENTER to bypass this prompt.

Output
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:CAjsV9M/tt5skazroTc1ZRGCBz+kGtYUIPhRvvZJYBs username@hostname
The key's randomart image is:
+---[RSA 3072]----+
|o   ..oo.++o ..  |
| o o +o.o.+...   |
|. . + oE.o.o  .  |
| . . oo.B+  .o   |
|  .   .=S.+ +    |
|      . o..*     |
|        .+= o    |
|        .=.+     |
|       .oo+      |
+----[SHA256]-----+

You now have a public and private key that you can use to authenticate. The next step is to place the public key on your server so that you can use SSH key authentication to log in.

There are multiple ways to upload your public key to your remote SSH server. The method you use depends largely on the tools you have available and the details of your current configuration.

The following methods all yield the same end result. The simplest, most automated method is described first, and the ones that follow it each require additional manual steps. You should follow these only if you are unable to use the preceding methods.

Copying Your Public Key Using ssh-copy-id

The simplest way to copy your public key to an existing server is to use a utility called ssh-copy-id. Because of its simplicity, this method is recommended if available.

The ssh-copy-id tool is included in the OpenSSH packages in many distributions, so you may already have it available on your local system. For this method to work, you must currently have password-based SSH access to your server.

To use the utility, you need to specify the remote host that you would like to connect to, and the user account that you have password-based SSH access to. This is the account where your public SSH key will be copied.

The syntax is:

ssh-copy-id username@remote_host

You may see a message like this:

Output
The authenticity of host '203.0.113.1 (203.0.113.1)' can't be established.
ECDSA key fingerprint is fd:fd:d4:f9:77:fe:73:84:e1:55:00:ad:d6:6d:22:fe.
Are you sure you want to continue connecting (yes/no)? yes

This means that your local computer does not recognize the remote host. This will happen the first time you connect to a new host. Type yes and press ENTER to continue.

Next, the utility will scan your local account for the id_rsa.pub key that we created earlier. When it finds the key, it will prompt you for the password of the remote user’s account:

Output
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
username@203.0.113.1's password:

Type in the password (your typing will not be displayed for security purposes) and press ENTER. The utility will connect to the account on the remote host using the password you provided. It will then copy the contents of your ~/.ssh/id_rsa.pub key into a file in the remote account’s home ~/.ssh directory called authorized_keys.

You will see output that looks like this:

Output
Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'username@203.0.113.1'"
and check to make sure that only the key(s) you wanted were added.

At this point, your id_rsa.pub key has been uploaded to the remote account. You can continue onto the next section.

Copying Your Public Key Using SSH

If you do not have ssh-copy-id available, but you have password-based SSH access to an account on your server, you can upload your keys using a conventional SSH method.

We can do this by outputting the content of our public SSH key on our local computer and piping it through an SSH connection to the remote server. On the other side, we can make sure that the ~/.ssh directory exists under the account we are using and then output the content we piped over into a file called authorized_keys within this directory.

We will use the >> redirect symbol to append the content instead of overwriting it. This will let us add keys without destroying previously added keys.

The full command will look like this:

cat ~/.ssh/id_rsa.pub | ssh username@remote_host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

You may see a message like this:

Output
The authenticity of host '203.0.113.1 (203.0.113.1)' can't be established.
ECDSA key fingerprint is fd:fd:d4:f9:77:fe:73:84:e1:55:00:ad:d6:6d:22:fe.
Are you sure you want to continue connecting (yes/no)? yes

This means that your local computer does not recognize the remote host. This will happen the first time you connect to a new host. Type yes and press ENTER to continue.

Afterwards, you will be prompted with the password of the account you are attempting to connect to:

Output
username@203.0.113.1's password:

After entering your password, the content of your id_rsa.pub key will be copied to the end of the authorized_keys file of the remote user’s account. Continue to the next section if this was successful.

Copying Your Public Key Manually

If you do not have password-based SSH access to your server available, you will have to do the above process manually.

The content of your id_rsa.pub file will have to be added to a file at ~/.ssh/authorized_keys on your remote machine somehow.

To display the content of your id_rsa.pub key, type this into your local computer:

cat ~/.ssh/id_rsa.pub

You will see the key’s content, which may look something like this:

~/.ssh/id_rsa.pub

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqql6MzstZYh1TmWWv11q5O3pISj2ZFl9HgH1JLknLLx44+tXfJ7mIrKNxOOwxIxvcBF8PXSYvobFYEZjGIVCEAjrUzLiIxbyCoxVyle7Q+bqgZ8SeeM8wzytsY+dVGcBxF6N4JS+zVk5eMcV385gG3Y6ON3EG112n6d+SMXY0OEBIcO6x+PnUSGHrSgpBgX7Ks1r7xqFa7heJLLt2wWwkARptX7udSq05paBhcpB0pHtA1Rfz3K2B+ZVIpSDfki9UVKzT8JUmwW6NNzSgxUfQHGwnW7kj4jp4AT0VZk3ADw497M2G/12N0PPB5CnhHf7ovgy6nL1ikrygTKRFmNZISvAcywB9GVqNAVE+ZHDSCuURNsAInVzgYo9xgJDW8wUw2o8U77+xiFxgI5QSZX3Iq7YLMgeksaO4rBJEa54k8m5wEiEE1nUhLuJ0X/vh2xPff6SQ1BL/zkOhvJCACK6Vb15mDOeCSq54Cr7kvS46itMosi/uS66+PujOO+xt/2FWYepz6ZlN70bRly57Q06J+ZJoc9FfBCbCyYH7U/ASsmY095ywPsBo1XQ9PqhnN1/YOorJ068foQDNVpm146mUpILVxmq41Cj55YKHEazXGsdBIbXWhcrRf4G2fJLRcGUr9q8/lERo9oxRm5JFX6TCmj6kmiFqv+Ow9gI0x8GvaQ== username@hostname

Access your remote host using whatever method you have available. This may be a web-based console provided by your infrastructure provider.

Once you have access to your account on the remote server, you should make sure the ~/.ssh directory is created. This command will create the directory if necessary, or do nothing if it already exists:

mkdir -p ~/.ssh

Now, you can create or modify the authorized_keys file within this directory. You can add the contents of your id_rsa.pub file to the end of the authorized_keys file, creating it if necessary, using this:

echo public_key_string >> ~/.ssh/authorized_keys

In the above command, substitute the public_key_string with the output from the cat ~/.ssh/id_rsa.pub command that you executed on your local system. It should start with ssh-rsa AAAA... or similar.

If this works, you can move on to test your new key-based SSH authentication.

If you have successfully completed one of the procedures above, you should be able to log in to the remote host without the remote account’s password.

The process is mostly the same:

ssh username@remote_host

If this is your first time connecting to this host (if you used the last method above), you may see something like this:

Output
The authenticity of host '203.0.113.1 (203.0.113.1)' can't be established.
ECDSA key fingerprint is fd:fd:d4:f9:77:fe:73:84:e1:55:00:ad:d6:6d:22:fe.
Are you sure you want to continue connecting (yes/no)? yes

This means that your local computer does not recognize the remote host. Type yes and then press ENTER to continue.

If you did not supply a passphrase for your private key, you will be logged in immediately. If you supplied a passphrase for the private key when you created the key, you will be required to enter it now. Afterwards, a new shell session will be created for you with the account on the remote system.

If successful, continue on to find out how to lock down the server.

If you were able to log in to your account using SSH without a password, you have successfully configured SSH key-based authentication for your account. However, password-based authentication is still active, meaning that your server is still exposed to brute-force attacks.

Before completing the steps in this section, make sure that you either have SSH key-based authentication configured for the root account on this server, or preferably, that you have SSH key-based authentication configured for an account on this server with sudo access. This step will lock down password-based logins, so ensuring that you will still be able to get administrative access is essential.

Once the above conditions are true, log in to your remote server with SSH keys, either as root or with an account with sudo privileges. Open the SSH daemon’s configuration file:

sudo nano /etc/ssh/sshd_config

Inside the file, search for a directive called PasswordAuthentication. This may be commented out. Uncomment the line by removing any # at the beginning of the line, and set the value to no. This will disable your ability to log in through SSH using account passwords:

/etc/ssh/sshd_config

PasswordAuthentication no

Save and close the file when you are finished. To actually implement the changes we just made, you must restart the service.

On most Linux distributions, you can issue the following command to do that:

sudo systemctl restart ssh

After completing this step, you’ve successfully transitioned your SSH daemon to only respond to SSH keys.

Hardware Security Modules (HSMs) provide an extra layer of security for SSH keys by keeping private keys stored in tamper-resistant hardware. Instead of storing private keys in a file, HSMs store them securely, preventing unauthorized access.

How to Use an HSM for SSH Authentication?

1. Check HSM Compatibility: Ensure your HSM supports SSH authentication.

2. Use a PKCS#11 Module: Most HSM integrations rely on a compatible PKCS#11 provider library supplied by the vendor (or by a tool like OpenSC). PKCS#11 is a cryptographic standard that defines an API for accessing cryptographic tokens such as HSMs, smart cards, and USB security keys.

3. Load the HSM-backed key into your SSH agent: On many systems you can ask ssh-agent to load identities from a PKCS#11 provider. The exact provider path varies by OS and vendor.

   ssh-add -s /usr/lib/opensc-pkcs11.so
   
   

4. Extract the public key and install it on the server: List the public keys exposed by the agent and add the relevant one to ~/.ssh/authorized_keys on the server.

   ssh-add -L
   
   

   You can redirect output to a file if you want, then copy that public key into authorized_keys:

   ssh-add -L > ~/.ssh/id_hsm.pub
   
   

5. Configure SSH to Use the HSM: Add the following to your SSH config file ~/.ssh/config:

   Host *
       IdentityAgent /run/user/1000/gnupg/S.gpg-agent.ssh
   

   Note: The path may vary depending on your system’s user ID.

   Now, SSH will automatically use the hardware-backed key for authentication.

Benefits of Using HSMs for SSH Key Management

• Enhanced Security: Private keys never leave the hardware.
• Protection from Theft: Even if the system is compromised, the key remains safe.
• Compliance: Useful for meeting security compliance requirements in regulated industries.

Even when SSH key authentication is configured correctly, small mistakes in file permissions, key placement, or SSH configuration can prevent successful login. If your SSH key login fails, the issue is usually related to one of the following common problems.

1. Incorrect File or Directory Permissions

SSH enforces strict permission rules for security reasons. If the permissions on your .ssh directory or key files are too open, the SSH server will refuse to use them.

On most OpenSSH servers, this behavior is controlled by the default StrictModes setting, which causes sshd to ignore keys if the account’s home directory, the ~/.ssh directory, or the authorized_keys file is writable by group or other users.

Check the permissions and ownership on the remote server:

ls -ld ~/.ssh
ls -l ~/.ssh/authorized_keys

Typical recommended permissions are:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Ensure that your private key has restrictive permissions. If the key is readable by other users, the SSH client may refuse to use it.

chmod 600 ~/.ssh/id_rsa

You should also ensure the directory and file are owned by the user account you are logging in as:

sudo chown -R username:username ~/.ssh

Also ensure your home directory is not writable by group or other users:

chmod 755 ~

If permissions are too permissive, SSH may silently ignore the authorized_keys file.

2. Public Key Not Properly Added to authorized_keys

A very common issue is copying the public key incorrectly. The key must be added as a single uninterrupted line inside the ~/.ssh/authorized_keys file on the server.

To verify the key:

cat ~/.ssh/authorized_keys

A valid key entry usually starts with the key type (algorithm) followed by a long base64-encoded blob and an optional comment, such as:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ... user@hostname

Modern keys may also look like this:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... user@hostname

Common mistakes include:

• Line breaks inserted into the key
• Extra spaces or missing characters
• Adding the private key instead of the public key
• Accidentally overwriting existing keys

If needed, copy the key again from your local machine:

ssh-copy-id username@remote_host

If you have multiple keys on your client, it is often safer to specify the exact public key file you want to install:

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@remote_host

3. Wrong Private Key Being Used

If you have multiple SSH keys, the SSH client may attempt to use the wrong one.

You can explicitly specify the key when connecting:

ssh -i ~/.ssh/id_rsa username@remote_host

If you want to ensure that SSH does not offer other identities from your agent, add IdentitiesOnly=yes:

ssh -i ~/.ssh/id_rsa -o IdentitiesOnly=yes username@remote_host

To see which keys SSH is trying, use verbose mode:

ssh -v username@remote_host

The debug output will show which keys are offered to the server and whether authentication succeeds or fails.

If you regularly use multiple keys, you can configure them in ~/.ssh/config:

Host myserver
    HostName remote_host
    User username
    IdentityFile ~/.ssh/id_rsa

4. SSH Service Not Restarted After Configuration Changes

If you disable password authentication or change SSH settings, you must restart the SSH daemon for the changes to take effect.

Before restarting, you can validate your server configuration for syntax errors with:

sudo sshd -t

On Ubuntu and Debian, the OpenSSH service is typically named ssh. On some other distributions (such as CentOS/RHEL/Fedora), it is commonly named sshd.

Restart SSH with one of the following commands:

sudo systemctl restart ssh

sudo systemctl restart sshd

If you are only making configuration changes and want to avoid dropping existing connections, you can often reload instead of restarting:

sudo systemctl reload ssh

If the service fails to restart, check logs for errors:

sudo journalctl -u ssh

sudo journalctl -u sshd

5. Incorrect Username or Host

Sometimes authentication fails simply because the SSH command uses the wrong username.

For example:

ssh root@remote_host

If the public key was added to another account, such as ubuntu or admin, authentication will fail.

Confirm which account has the key installed:

cat /home/username/.ssh/authorized_keys

Then connect using that username.

6. SSH Agent Not Loaded with the Private Key

If you use a passphrase-protected key, you may need to add the key to the SSH agent.

Start the agent:

eval "$(ssh-agent -s)"

Add your key:

ssh-add ~/.ssh/id_rsa

You can confirm that your key is loaded by listing identities in the agent:

ssh-add -l

This allows the agent to store the decrypted key and automatically authenticate future connections.

7. Server Still Allowing Password Authentication

If your SSH client prompts for a password, it usually means one of two things: either password authentication is enabled on the server, or public key authentication failed and SSH is falling back to another allowed method.

Open the SSH configuration file:

sudo nano /etc/ssh/sshd_config

Ensure this setting is present:

PasswordAuthentication no

If you want to fully disable password-like interactive prompts, you can also ensure that keyboard-interactive authentication is disabled:

KbdInteractiveAuthentication no

You should also confirm that public key authentication is enabled:

PubkeyAuthentication yes

Then reload or restart SSH:

sudo systemctl restart ssh

Be careful when disabling password authentication. Make sure SSH keys are working correctly before applying this change, or you could lock yourself out of the server.

8. Firewall or Network Restrictions

If you cannot connect at all, the issue may not be related to SSH keys but to networking.

Check whether the SSH server is listening on a port (commonly 22 unless you changed it):

sudo ss -tlnp | grep ssh

If a firewall is enabled, ensure SSH is allowed:

sudo ufw allow ssh
sudo ufw reload

You should also verify that you are connecting to the correct IP address and port.

Debugging Tip: Use SSH Verbose Mode

When troubleshooting SSH authentication issues, the most useful diagnostic tool is verbose output:

ssh -vvv username@remote_host

This command shows detailed logs of the authentication process, including:

• Which keys the client attempts to use
• Whether the server accepts or rejects them
• Configuration issues that may prevent you from logging in

These logs often reveal exactly where the authentication process is failing.

If you have access to the server through a console or another session, server-side logs are also extremely helpful. On many Debian/Ubuntu systems you can check /var/log/auth.log, and on systemd-based systems you can inspect the SSH unit logs:

sudo tail -n 50 /var/log/auth.log

sudo journalctl -u ssh -n 50 --no-pager

1. How do I generate an SSH key in Linux?

To generate an SSH key in Linux, use the ssh-keygen command in your terminal. By default, this will create an RSA key pair:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

For a more detailed step-by-step guide, refer to our tutorial on How To Set Up SSH Keys.

2. What is ssh-keygen in Linux?

ssh-keygen is a command-line tool used to generate, manage, and convert SSH keys. It allows you to create secure authentication credentials for remote access. You can learn more about ssh-keygen and how it works in How to Create SSH Keys with OpenSSH on macOS or Linux.

3. How to generate an SSH-2 RSA key on Linux?

SSH-2 is the current standard protocol for SSH authentication. To generate an SSH-2 RSA key, run:

ssh-keygen -t rsa -b 4096

You can check more details on adding SSH keys in our official documentation on How to Add SSH Keys.

4. How to create a valid SSH key?

A valid SSH key should meet these criteria:

• Use a strong key type, such as RSA (4096-bit) or Ed25519.
• Ensure the private key is securely stored and protected.
• Use a passphrase for added security.
• Avoid using weak algorithms like DSA.
• For a deep dive into SSH encryption and security, check out Understanding the SSH Encryption and Connection Process.

5. How to generate an SSH key from the terminal?

Simply run:

ssh-keygen

This will generate a public and private key pair, usually stored in ~/.ssh/.

6. How to generate a private key?

The ssh-keygen command automatically generates a private key. The private key is typically stored at:

~/.ssh/id_rsa

or for newer standards:

~/.ssh/id_ed25519

Keep this file secure and do not share it.

7. What is the difference between public and private SSH keys?

8. How to disable password authentication on a Linux server?

To enhance security, disable password authentication by editing the SSH configuration file:

sudo nano /etc/ssh/sshd_config

Find the line:

PasswordAuthentication yes

Change it to:

PasswordAuthentication no

Then restart SSH:

sudo systemctl restart ssh

For further instructions, refer to How to Create SSH Keys with OpenSSH on macOS or Linux.

You should now have SSH key-based authentication configured and running on your server, allowing you to sign in without providing an account password. From here, there are many directions you can head. If you’d like to learn more about working with SSH, take a look at our following tutorials:

• SSH essentials guide
• Create SSH Keys with OpenSSH on macOS, Linux, or Windows
• How to Use SSH to Connect to a Remote Server (Step-by-Step Guide)
• Understanding the SSH Encryption and Connection Process