Ruby gem to inspect web pages. It scrapes a given URL and returns its title, description, meta tags, links, images, and more.
Add this line to your application's Gemfile:
gem 'webinspector'And then execute:
$ bundle
Or install it yourself as:
$ gem install webinspector
page = WebInspector.new('http://example.com')page = WebInspector.new('http://example.com', {
timeout: 30, # Request timeout in seconds (default: 30)
retries: 3, # Number of retries (default: 3)
headers: {'User-Agent': 'Custom UA'} # Custom HTTP headers
})page.response.status # 200
page.response.headers # { "server"=>"apache", "content-type"=>"text/html; charset=utf-8", ... }
page.status_code # 200
page.success? # true if the page was loaded successfully
page.error_message # returns the error message if anypage.url # URL of the page
page.scheme # Scheme of the page (http, https)
page.host # Hostname of the page (like, example.com, without the scheme)
page.port # Port of the page
page.title # title of the page from the head section
page.description # description of the page
page.links # array of all links found on the page (absolute URLs)
page.images # array of all images found on the page (absolute URLs)
page.meta # meta tags of the page
page.favicon # favicon URL if availablepage.meta # all meta tags
page.meta['description'] # meta description
page.meta['keywords'] # meta keywords
page.meta['og:title'] # OpenGraph titlepage.domain_links('example.com') # returns only links pointing to example.com
page.domain_images('example.com') # returns only images hosted on example.compage.find(["ruby", "rails"]) # returns [{"ruby"=>3}, {"rails"=>1}]page.to_hash # returns a hash with all page data- Steven Shelby (@stevenshelby)
- Sam Nissen (@samnissen)
The WebInspector gem is released under the MIT License.
- Fork it ( https://github.com/davidesantangelo/webinspector/fork )
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create a new Pull Request