Focused crawls are collections of frequently-updated webcrawl data from narrow (as opposed to broad or wide) web crawls, often focused on a single domain or subdomain.
pip install shub
shub login
Insert your Scrapinghub API Key: <API_KEY># Deploy the spider to Scrapy Cloud shub deploy
# Schedule the spider for execution shub schedule blogspider
Spider blogspider scheduled, watch it running here:
https://app.scrapinghub.com/p/26731/job/1/8# Retrieve the scraped data shub items 26731/1/8
{"title":"Improved Frontera: Web Crawling at Scale with Python 3 Support"}{"title":"How to Crawl the Web Politely with Scrapy"}...