In a recent business endeavor, I found it necessary to collect bulk data from different online sources in order to centralize it and make it easier for people to find and to make sense of. I would have liked if those sites had exposed public APIs, but unfortunately none did. So, I decided to try my hand at a little web crawling and scraping to obtain this metadata and after searching around for awhile, I found that many people when faced with the same issue have turned towards Python. After all, if it was good enough for Google, it's definitely good enough for me!

In this course I will share some of my findings and show you how you can go about creating your own basic web crawler and scraper.

The full project source can be found in the itunes.py file.

These are source files for the Tuts+ course: Crawl the Web With Python

Available on Tuts+. Teaching skills to millions worldwide.