Authentication and python Requests

Question

I am trying to download some documents using requests, but the page is redirecting me to a userlog in screen and therefor downloading the HTML page.

I've tried doing:

c=requests.get(url,auth=HTTPBasicAuth('user','pass'))

But I'm not getting authenticated.

I've also tried vanilla and Digest.

The form itself looks like this:

<input id="username" name="username" class="required" tabindex="1" type="text" value="" size="25" autocomplete="false"/>
<br/>

<label for="password">Password</label>
<input id="password" name="password" class="required" tabindex="2" type="password" value="" size="25" autocomplete="off"/>

Do I need to pass in the username and password as a part of the payload? If so, how do I do that? I've tried a few different ways so far.

is it basic authentication, do you get a browser pop up asking for username and password, or is it a regular page with a textfield requesting username and password? — Samy Vilar
– Samy Vilar, Commented Sep 12, 2012 at 23:36
@samy.vilar I believe basic auth. I attempt to download a document and it redirects to another page with a simple form (partially shown above) — James R
– James R, Commented Sep 13, 2012 at 13:00

James R · Accepted Answer · 2012-09-13 17:26:17Z

7

Basically, it had to do with grabbing the authentication ID off the page and passing in cookies.

This is basically what I did:

from bs4 import BeautifulSoup as bs
import requests
s = requests.session()
url = r'url_i_care_about'

def authenticate(s, url):
    headers = {'username': 'myuser', 'password': 'mypasss', '_Id': 'submit'}
    page=s.get(url)
    soup=bs(page.content)
    value=soup.form.find_all('input')[2]['value']
    headers.update({'value_name':value})
    auth = s.post(url, params=headers, cookies=page.cookies)

answered Sep 13, 2012 at 17:26

James R

4,6564 gold badges33 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Authentication and python Requests

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related