How to send a GET request with headers via python

Question

I got fiddler to capture a GET request, I want to re send the exact request with python. This is the request I captured:

GET https://example.com/api/content/v1/products/search?page=20&page_size=25&q=&type=image HTTP/1.1
Host: example.com
Connection: keep-alive
Search-Version: v3
Accept: application/json
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Referer: https://example.com/search/?q=&type=image&page=20
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9

You can try transmitting the HTTP request as-is using a regular TCP socket — Omer Tuchfeld
– Omer Tuchfeld, Commented Mar 19, 2019 at 11:21
You often have to export requests from Fiddler to Python you might want to use the Export Sessions function which can generate a curl command from a Fiddler request. Then you can convert this curl command e.g. using this converter to Python code. Alternatively there is a Fiddler extension that directly generated Python code: chadsowald.com/software/fiddler-extension-request-to-code — Robert
– Robert, Commented Mar 20, 2019 at 19:17

summer · Accepted Answer · 2019-03-19 13:16:55Z

You can use the requests module.

The requests module automatically supplies most of the headers for you so you most likely do not need to manually include all of them.

Since you are sending a GET request, you can use the params parameter to neatly form the query string.

Example:

import requests

BASE_URL = "https://example.com/api/content/v1/products/search"

headers = {
    "Connection": "keep-alive",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
}

params = {
    "page": 20,
    "page_size": 25,
    "type": "image"
}

response = requests.get(BASE_URL, headers=headers, params=params)

Gokhan Gerdan · Accepted Answer · 2019-03-19 11:58:21Z

import requests

headers = {
    'authority': 'stackoverflow.com',
    'cache-control': 'max-age=0',
    'upgrade-insecure-requests': '1',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36',
    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
    'referer': 'https://stackoverflow.com/questions/tagged/python?sort=newest&page=2&pagesize=15',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.9,tr-TR;q=0.8,tr;q=0.7',
    'cookie': 'prov=6bb44cc9-dfe4-1b95-a65d-5250b3b4c9fb; _ga=GA1.2.1363624981.1550767314; __qca=P0-1074700243-1550767314392; notice-ctt=4%3B1550784035760; _gid=GA1.2.1415061800.1552935051; acct=t=4CnQ70qSwPMzOe6jigQlAR28TSW%2fMxzx&s=32zlYt1%2b3TBwWVaCHxH%2bl5aDhLjmq4Xr',
}

response = requests.get('https://stackoverflow.com/questions/55239787/how-to-send-a-get-request-with-headers-via-python', headers=headers)

This is an example of how to send a get request to this page with headers.

Mikhail Vladimirov · Accepted Answer · 2019-03-19 11:22:42Z

0

You may open SSL socket (https://docs.python.org/3/library/ssl.html) to example.com:443, write your captured request into this socket as raw bytes, and then read HTTP response from the socket.

You may also try to use http.client.HTTPResponse class to read and parse HTTP response from your socket, but this class is not supposed to be instantiated directly, so some unexpected obstacles could emerge.

answered Mar 19, 2019 at 11:22

Mikhail Vladimirov

13.9k1 gold badge40 silver badges41 bronze badges

Collectives™ on Stack Overflow

How to send a GET request with headers via python

3 Answers 3

Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Related