Create agents that monitor and act on your behalf. Your agents are standing by!

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Web Scraper in Go, similar to BeautifulSoup

Creating Scrapy scrapers via the Django admin interface

🥫 The simple, fast, and modern web scraping library

Take the hassle out of web scraping

a class that uses scraped proxies to make http GET/POST requests (Python requests)

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

An R web crawler and scraper

Be nice on the web

🗽 A Simple Demonstration of the New York Times App 📱 using Jsoup web crawler with MVVM Architecture 🔥

LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping

Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.

Extract price and indicator data from TradingView charts to create ML datasets

Powerful and flexible Instagram scraping library for Python, providing easy-to-use and expressive tools for accessing data programmatically

Open Source web scraping API. Falkor turns web pages into queryable JSON

📲 Bot to help solve HQ trivia

An extensible API for breaking captchas

This repository contains all the code I use in my YouTube tutorials.

Github stargazers information gathering tool

ralger makes it easy to scrape a website. Built on the shoulders of titans: rvest, xml2.

operating systems three easy pieces by Rezmi

🎬 A Crunchyroll show/season ripper

Scrapes g4g and creates PDF

An exploration of New York Times crossword answers from 1994-2017, i.e. the Will Shortz era.

A php crawler that finds emails on the internets

A TikTokBot that downloads trending tiktok videos and compiles them using FFmpeg

Code for the second edition Web Scraping with Python book by Packt Publications

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c