Which public key format should be used for libssh2 public key authentication?

Question

I am trying pass publickey (id_rsa.pub) for sftp_write example in libssh2 library.

const char *pubkey = "/home/username/.ssh/id_rsa.pub";
const char *privkey = NULL;

if(libssh2_userauth_publickey_fromfile(session, username,

                                       pubkey, privkey,
                                       password)) {
    fprintf(stderr, "\tAuthentication by public key failed\n");
    goto shutdown;
}

I have tried openssh format and standard ssh format keys as below.

I got the below errors for two formats of the

• standard ssh format, Athentication failed: cannot open the file -16

  ---- BEGIN SSH2 PUBLIC KEY ----
  AAAAB3NzaC1yc2EAAAADAQABAAABAQCjnPF5vKg7NuUXkGbK0Nh5d3GjLNXHB+VozgVn5Ego
  DVSjwezr30YJm8gzGrk5aUAEJKbWSUczVj9vCrn7O54K1/VXOggC1a6pun3+Iqvuv0BINRj3
  YuC+c9s803mYPSpzZpNPFf3Uj49lKrroUIdnEz3qo/Z9kFbpWfClJPlb7xjrZaZkSgolbxih
  8gZ0w3iAuMR5nwnEGQnHkk751jxOtYopzT4w31gFmFYcq/LRbP4S2xJ0r8wl58J8Mm/U/yZx
  EhDARHx/mkqMILofAqFvsFSaaASUuFmB9uUJsgJ69s9FbgXnpGhyGoH0nyVsdM/oqwIXtJji
  IvbVBEomnCrX
  ---- END SSH2 PUBLIC KEY ----
  

• Open SSH Format, Authentication failed

     LIBSSH2_ERROR_PUBLICKEY_UNRECOGNIZED \ 
     LIBSSH2_ERROR_AUTHENTICATION_FAILED ,-18
  

  "ssh-rsa" AAAAB3NzaC1yc2EAAAADAQABAAABAQCjnPF5vKg7NuUXkGbK0Nh5d3GjLNXHB+VozgVn5EgoDVSjwezr30YJm8gzGrk5aUAEJKbWSUczVj9vCrn7O54K1/VXOggC1a6pun3+Iqvuv0BINRj3YuC+c9s803mYPSpzZpNPFf3Uj49lKrroUIdnEz3qo/Z9kFbpWfClJPlb7xjrZaZkSgolbxih8gZ0w3iAuMR5nwnEGQnHkk751jxOtYopzT4w31gFmFYcq/LRbP4S2xJ0r8wl58J8Mm/U/yZxEhDARHx/mkqMILofAqFvsFSaaASUuFmB9uUJsgJ69s9FbgXnpGhyGoH0nyVsdM/oqwIXtJjiIvbVBEomnCrX
  

Can some one suggest if its the public key format problem.

I am using Bitvise server on windows for testing the libssh2 client publickey authentication.

Answer 1

It seems you are not passing the private key to libssh2.

SSH pubkey authentication requires two machines, a server and client. The client should generate a private and public keypair, where the public key should be copied to the server using any method available prior to pubkey authentication attempts.The private key should never leave the client.

When connecting to the server using the client, one should use the private key. Authenticating using only the public key only will not work, as you can only encrypt the negotiation packets, but never actually decrypt them. Reading through the documentation also states that the private key is a must, while the public key might be set to NULL in some cases.

Reproducing the problem

I've tried reproducing the error, but it seems to work for me, below I will describe how I did it, and some speculations regarding your problem:

1. Made a directory and copied in the example file sftp_write.c from libssh2.
2. Changed auth_pw from 1 to 0 on line 27.
3. Changed pubkey and privkey to use my own keys, and added debugging.

    const char *pubkey = "/home/user/.ssh/id_rsa_PEM.pub";
    const char *privkey = "/home/user/.ssh/id_rsa_PEM";
    int liberr;
    if(liberr = libssh2_userauth_publickey_fromfile(session, username,
                                               pubkey, privkey,
                                               password)) {
        fprintf(stderr, "\tAuthentication by public key failed: %d\n", liberr);
        goto shutdown;
    }

4. changed

#include "libssh2_config.h"
#include <libssh2.h>
#include <libssh2_sftp.h>

#ifdef HAVE_WINSOCK2_H
# include <winsock2.h>
#endif
#ifdef HAVE_SYS_SOCKET_H
# include <sys/socket.h>
#endif
#ifdef HAVE_NETINET_IN_H
# include <netinet/in.h>
#endif
# ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
#ifdef HAVE_ARPA_INET_H
# include <arpa/inet.h>
#endif

to

#include <libssh2.h>
#include <libssh2_sftp.h>

#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <arpa/inet.h>

To match my setup.

5. Used Docker to compile the image by running

docker run -i --rm \
    -w $(pwd) -v $(pwd):$(pwd) \
    -v /etc/passwd:/etc/passwd:ro  \
    -v /etc/group:/etc/group:ro \
    -u $(id -u):$(id -u) \
    datafr/libssh2:latest \
    gcc -g -I /usr/include/ -L /usr/lib/ -l ssh2 sftp_write.c -o a.out

6. Ran the program with

./a.out 127.0.0.1 $USER

Results

Running this code results in error -19: LIBSSH2_ERROR_PUBLICKEY_UNVERIFIED. This is because the docker image uses an older version of libssh2 which requires older PEM keys. This is fixed by creating a new keypair with

ssh-keygen -m PEM -t rsa -P "" -f id_rsa_PEM

And modifying the code to use the new keys.

Running the code once more gives us error -18: LIBSSH2_ERROR_AUTHENTICATION_FAILED. This is fixed by adding our new public key id_rsa_PEM.pub to the servers authorized_keys-file.

After this, we get no more errors, and the output looks like this:

$ ./a.out 127.0.0.1 $USER
Fingerprint: XXXXXXX
libssh2_sftp_init()!
libssh2_sftp_open()!
libssh2_sftp_open() is done, now send data!
all done

And I can confirm the creation of a file called /tmp/TEST containing our code from sftp_write.c.

Further speculations

I've noticed that error -16: LIBSSH2_ERROR_FILE occurs when the file is either non-existent or has the wrong permissions, so I'll ask you to double check your file permissions. I'll post my own permissions as reference.

Client ~/.ssh/ directory:

$ ls -alh
total 28K
drwx------  2 user user 4,0K Sep 28 15:37 .
drwxr-xr-x 32 user user 4,0K Sep 28 15:36 ..
-rw-rw-r--  1 user user 2,7K Sep 28 15:36 config
-rw-------  1 user user 2,5K Sep 28 15:36 id_rsa_PEM
-rw-r--r--  1 user user  574 Sep 28 15:36 id_rsa_PEM.pub
-rw-------  1 user user 6,3K Sep 28 15:36 known_hosts

Server ~/.ssh/ directory:

$ ls -alh
total 20K
drwx------  2 user user 4,0K Sep 28 15:39 .
drwxr-xr-x 32 user user 4,0K Sep 28 15:39 ..
-rw-------  1 user user 3,4K Sep 28 15:39 authorized_keys