Data transfer with curl

Welcome to the next pikoTutorial !

Short for "Client URL," curl is a command-line tool and library for transferring data with URLs. It supports various protocols, including HTTP, HTTPS, FTP, FTPS, SCP, SFTP, LDAP, and more.

Basic usage

At its core, curl allows you to fetch content from or send content to a server. Here’s how you can use it for basic data retrieval:

curl [URL]

Replace [URL] with the actual URL you want to retrieve data from. For example:

curl https://example.com

This command fetches the HTML content from https://example.com and displays it in the terminal.

Handling HTTP methods

curl supports different HTTP methods, which are essential for interacting with web services beyond simple GET requests. To specify a different HTTP method (such as POST, PUT, DELETE), use the -X flag:

curl -X POST -d "key1=value1&key2=value2" https://api.example.com/resource

In this example, -X POST specifies that a POST request should be made, and -d "key1=value1&key2=value2" sends data as part of the request body.

Handling headers

HTTP headers can convey crucial information or authentication credentials. curl allows you to set headers using the -H flag:

curl -H "Content-Type: application/json" -H "Authorization: Bearer <token>" https://api.example.com/resource

Replace token with your actual authentication token. Multiple headers can be specified by repeating the -H flag.

Handling file transfers

curl can also be used for uploading and downloading files. To upload a file:

curl -X POST -F "[email protected]" https://example.com/upload

In this example, -F "[email protected]" sends the contents of localfile.txt as a file upload.

Output options

By default, curl outputs the fetched content to standard output (usually the terminal). You can save this output to a file using the -o or -O flags:

curl -o filename.html https://example.com/page.html

The -o flag followed by a filename saves the output to a specific file. Alternatively, using -O saves the output to a file with the same name as in the URL.

Verbose mode

For debugging or understanding the details of a request, use the -v or --verbose flag:

curl -v https://example.com

This command provides detailed information about the request and response headers.