#
crawl
Here are 175 public repositories matching this topic...
INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱🧰 ,旨在安全快捷的帮助用户拿回自己的数据,工具代码开源,流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。
-
Updated
Feb 8, 2021 - Python
The A11y Machine is an automated accessibility testing tool which crawls and tests pages of any web application to produce detailed reports.
-
Updated
Dec 17, 2019 - JavaScript
腾讯新闻、知乎话题、微博粉丝,Tumblr爬虫、斗鱼弹幕、妹子图爬虫、分布式设计等
python
redis
golang
awesome
tumblr
websockets
zhihu
crawl
scrapy
weibo
tencent
douyu
scrapy-redis
tumblr-bot
-
Updated
Apr 9, 2020 - Python
Bitextor generates translation memories from multilingual websites.
crawler
translation
dictionaries
tokenizer
wget
crawl
apertium
warc
tmx
corpus-generator
httrack
sentence-segmentation
corpus-tools
creepy
corpus-processing
hunalign
parallel-corpora
document-aligner
lett
bicleaner
-
Updated
Feb 2, 2021 - Python
A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.
-
Updated
Dec 7, 2020 - Shell
[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library
nlp
bot
php
machine-learning
scraper
ai
scraping
crawling
artificial-intelligence
crawl
scrape
scraped-data
diffbot
-
Updated
Jul 4, 2018 - PHP
爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer
python
chrome-extension
crawler
scraper
awesome
spider
scraping
crawl
awesome-list
chrome-extensions
-
Updated
Sep 18, 2019
A Moodle Crawler that downloads course content from Moodle (eg. lecture pdfs)
content
crawler
assets
download
dhbw
crawl
moodle
downloads
moodle-crawler
donwnloader
moodle-downloader
assets-downloader
moodle-downlaader
moodle-download
-
Updated
Sep 3, 2020 - Python
Improve this page
Add a description, image, and links to the crawl topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the crawl topic, visit your repo's landing page and select "manage topics."


Is there an option to crawl events out of Facebook?
If not, would it be easy to implement? I could assist if there is interest for that.