SpiderClub

Repositories

weibospider

⚡ A distributed crawler for weibo, building with celery and requests.

python3 data-analysis weibo sina distributed-crawler weibospider

Python MIT 1,238 4,412 9 (2 issues need help) 2 Updated Jul 11, 2020

haipproxy

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

redis crawler spider scheduler distributed scrapy high-availability

Python MIT 840 4,514 30 1 Updated Nov 1, 2019

smart_login

各大网站登陆方式，有的是通过selenium登录，有的是通过抓包直接模拟登录（精力原因，目前不再继续维护）

Python MIT 384 993 14 (1 issue needs help) 0 Updated Apr 9, 2018