How to download protected PDF (ViewDocument) using Selenium or requests?

Question

I'm trying to download a protected PDF from the New York State Courts NYSCEF website using Python. The URL looks like this:

https://iapps.courts.state.ny.us/nyscef/ViewDocument?docIndex=cdHe_PLUS_DaUdFKcTLzBtSo6zw==

When I try to use requests.get() or even navigate to the page with Selenium, I either get:

• A 403 Forbidden response (via requests)
• Or a blank page with no <embed> tag (via Selenium)

Here’s what I’ve tried:

Using requests:

import requests

url = "https://iapps.courts.state.ny.us/nyscef/ViewDocument?docIndex=..."
headers = {
    "User-Agent": "Mozilla/5.0",
    "Referer": "https://iapps.courts.state.ny.us/nyscef/"
}
response = requests.get(url, headers=headers)
print(response.status_code)  # Always 403

And using SeleniumBase:

from seleniumbase import SB

with SB(headless=False) as sb:
    sb.open(url)
    sb.wait(5)
    try:
        embed = sb.find_element("embed")
        print(embed.get_attribute("src"))
    except Exception as e:
        print("❌ No embed tag found", e)

Nothing works.

Full code for reference:

from seleniumbase import SB
import requests
import os
import time

def download_pdf_with_selenium_and_requests():
    # Target document URL
    doc_url = "https://iapps.courts.state.ny.us/nyscef/ViewDocument?docIndex=cdHe_PLUS_DaUdFKcTLzBtSo6zw=="

    # Setup download directory
    download_dir = os.path.join(os.getcwd(), "downloads")
    os.makedirs(download_dir, exist_ok=True)
    filename = os.path.join(download_dir, "NYSCEF_Document.pdf")

    with SB(headless=True) as sb:
        # Step 1: Navigate to the document page (using browser session)
        sb.open(doc_url)
        time.sleep(5)  # Wait for any redirects/cookies to be set

        # Step 2: Grab the actual PDF <embed src>
        try:
            embed = sb.find_element("embed")
            pdf_url = embed.get_attribute("src")
            print(f"Found PDF URL: {pdf_url}")
        except Exception as e:
            print(f"No <embed> tag found: {e}")
            return

        # Step 3: Extract cookies from Selenium session
        selenium_cookies = sb.driver.get_cookies()
        session = requests.Session()
        for cookie in selenium_cookies:
            session.cookies.set(cookie['name'], cookie['value'])

        # Step 4: Download PDF using requests with cookies
        headers = {
            "User-Agent": "Mozilla/5.0",
            "Referer": doc_url
        }

        response = session.get(pdf_url, headers=headers)
        if response.status_code == 200 and "application/pdf" in response.headers.get("Content-Type", ""):
            with open(filename, "wb") as f:
                f.write(response.content)
            print(f"PDF saved as: {filename}")
        else:
            print(f"PDF download failed. Status: {response.status_code}")
            print(f"Content-Type: {response.headers.get('Content-Type')}")
            print(f"Final URL: {response.url}")

if __name__ == "__main__":
    download_pdf_with_selenium_and_requests()

Response:

No <embed> tag found: Message: 
 Element {embed} was not present after 10 seconds!

Answer 1

With SeleniumBase, you can do the following to download that file to the ./downloaded_files/ folder:

from seleniumbase import SB

with SB(uc=True, test=True, external_pdf=True) as sb:
    url = "https://iapps.courts.state.ny.us/nyscef/ViewDocument?docIndex=cdHe_PLUS_DaUdFKcTLzBtSo6zw=="
    sb.activate_cdp_mode(url)
    sb.sleep(10)

Answer 2

PDF's by their very nature are binary files and thus have no native protection unless the source requires a release code to allow download or the viewer includes a DRM method. Alternatively, one method is to scramble (encrypt) the PDF. So an opener password is needed to unscramble the key PDF readers functionality.

This is why many sites include a challenger mechanism, like a region IP block, cookies, captchas or timer-based limitation.

Likewise, some sites / PDF files may even in the past have tested for Adobe DRM Reader is the viewer.

Since a PDF must download to view the security is often introduced by cookies testing for browsers. Also the URL is not a /file.pdf but an instructive URL. Thus, without Agent cookies or a true filename a PDF reader can not download a file.

The given example is a Public file and in its response has

Content-Disposition: inline; filename="2025_1214_Citizens_Bank_N_A_v_Jose_L_Benitez_SR_et_al_NOTICE_OF_PENDENCY_6.pdf"

This indicates it is not protected at all but does expect a browsers signature (user Agent) to allow for "inline" Dispositioning from server to recipient.

Thus to easily download and view you just need to signal as a browser user. NOTE: This shows that in this case cookies and other automatic challenge bypass is not needed the file is NOT protected, however, may not work for more complex cases.

cURL -Lo file.pdf -A "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/81.0" https://iapps.courts.state.ny.us/nyscef/ViewDocument?docIndex=cdHe_PLUS_DaUdFKcTLzBtSo6zw==