Introduction

When managing multiple Linux servers, frequent SSH logins can become repetitive and time-consuming — especially when automating tasks, running scripts remotely, or setting up services like Ansible. A passwordless SSH connection offers a secure and convenient way to streamline server communication without compromising on security.

In this guide, we’ll walk through how to set up passwordless SSH authentication between two Linux servers using SSH key-based authentication. This method eliminates the need to manually enter a password every time you connect from one server to another. Instead, it uses a public-private key pair to authenticate access securely.

Whether you’re a system administrator, DevOps engineer, or just someone managing a few Linux boxes, this setup will make your workflow more efficient and secure.

What is SSH?

SSH (Secure Shell) is a cryptographic network protocol used to securely connect to remote systems over an unsecured network. It enables users to access and control remote machines, typically via a command-line interface, while encrypting all communications to protect against eavesdropping, tampering, and impersonation.

Key Features:

• Encrypted communication
• Remote command execution
• Tunnel creation for port forwarding

Prerequisites

• Two Linux servers (Server A and Server B).
• Administrative access to both servers.
• AWS Account.

Step 1 : Create 2 Linux Servers from EC2

1 . Provide a name for the instance.

2 . Give number of instances as 2, because we are gonna need 2 servers to interact with and without 2 servers the passwordless connection won’t be possible.

1. Select Amazon Linux 2023 AMI or Ubuntu AMI based on your convenience.

2. Select the required key pair.

3. Click on Launch Instance , after setting up the instance.

Step 2 : Generate SSH Key Pair on Server-A

1. Connect the server1 using SSH or Physical accesss.
2. Provide a name for Server-A as Server1 using the command

    sudo hostnamectl set-hostname server1

1. Generate an SSH key pair by running the following command on Server1, then Click enter 3 times for confirming the generation of the key — pair.

1. After generating the key pair on server1 , there would be 2 key-pairs known as Public-key (id_rsa.pub) and Private_key (id_rsa).

2. To view the content inside of both the keys , then enter the following command :

Step 3: Copy the Public key into the Server B

1. Change the hostname of the Server B to Server2 using the above shown command .
2. Copy the Public key content and paste it into the Server2. The Public key must be pasted into a authorized_keys which allows passwordless SSH from the machine which holds the corresponding Private-key. The command is :

echo "public key of server1" >> authorized_keys

1. The correct example of the above command is given below :

1. After pasting the Public-key of server1 into server2 , then go to server1 for accessing server2 from server1.

Step 4 : Test the Passwordless Connection

Attempt to access Server2 from Server1 :

1. In order to access server 2 from server1 you need to enter or run a command on server1(Server A):

1. So the actual command of accessing the server2 from server1 and the command must be run on server1 itself or else the connection would not happen. Run (ssh [email protected]) this on server1, 13.203.67.192 is the public IP of Server 2 .

Troubleshooting Tips

If the passwordless connection doesn’t work, here are some troubleshooting steps:

1. Check Permissions: Ensure the .ssh directory on both Server A and Server B has the correct permissions. It should be owned by the user and have restricted permissions:

chmod 700 ~/.ssh 
chmod 600 ~/.ssh/authorized_keys

1. Key File Names: Verify that you are using the default key names (id_rsa and id_rsa.pub) or the names you specified during key generation.
2. SSH Agent: Make sure you have added the private key to the SSH agent on Server A using ssh-add:

ssh-add ~/.ssh/id_rsa

Conclusion

Setting up a passwordless SSH connection between two Linux servers is a simple yet powerful way to streamline secure access, automate tasks, and improve system administration efficiency. By generating an SSH key pair and copying the public key to the remote server, you eliminate the need to enter a password each time you connect — without compromising on security.