Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Permalink
Checking mergeability…
Don’t worry, you can still create the pull request.
Comparing changes
Choose two branches to see what’s changed or to start a new pull request.
If you need to, you can also .
Open a pull request
Create a new pull request by comparing changes across two branches. If you need to, you can also .
- 2 commits
- 2 files changed
- 0 comments
- 2 contributors
Commits on Jan 29, 2015
Commits on Apr 01, 2015
Unified
Split
Showing
with
36 additions
and 11 deletions.
- +9 −0 lib/linguist/heuristics.rb
- +27 −11 test/test_heuristics.rb
| @@ -23,6 +23,15 @@ def self.call(blob, languages) | ||
| [] # No heuristics matched | ||
| end | ||
|
|
||
| # Public: Do we have a heuristic for these candidate languages? | ||
| # | ||
| # languages - Array of Language objects | ||
| # | ||
| # Returns true if there are matching heuristics or nil | ||
| def self.defined_for?(languages) | ||
| @heuristics.select { |h| h.matches?(languages) }.any? | ||
| end | ||
|
|
||
| # Internal: Define a new heuristic. | ||
| # | ||
| # languages - String names of languages to disambiguate. | ||
| @@ -22,6 +22,17 @@ def test_no_match | ||
| assert_equal [], results | ||
| end | ||
|
|
||
| def assert_heuristics(hash) | ||
| candidates = hash.keys.map { |l| Language[l] } | ||
|
|
||
| hash.each do |language, blobs| | ||
| Array(blobs).each do |blob| | ||
| result = Heuristics.call(file_blob(blob), candidates) | ||
| assert_equal [Language[language]], result, "Failed for #{blob}" | ||
| end | ||
| end | ||
| end | ||
|
|
||
| # Candidate languages = ["C++", "Objective-C"] | ||
| def test_obj_c_by_heuristics | ||
| # Only calling out '.h' filenames as these are the ones causing issues | ||
| @@ -148,17 +159,6 @@ def test_cs_by_heuristics | ||
| }) | ||
| end | ||
|
|
||
| def assert_heuristics(hash) | ||
| candidates = hash.keys.map { |l| Language[l] } | ||
|
|
||
| hash.each do |language, blobs| | ||
| Array(blobs).each do |blob| | ||
| result = Heuristics.call(file_blob(blob), candidates) | ||
| assert_equal [Language[language]], result, "Failed for #{blob}" | ||
| end | ||
| end | ||
| end | ||
|
|
||
| def test_ls_by_heuristics | ||
| assert_heuristics({ | ||
| "LiveScript" => "LiveScript/hello.ls", | ||
| @@ -172,4 +172,20 @@ def test_ts_by_heuristics | ||
| "XML" => all_fixtures("XML", "*.ts") | ||
| }) | ||
| end | ||
|
|
||
| def output_languages(languages) | ||
| languages.map { |l| l.name }.join(", ") | ||
| end | ||
|
|
||
| # If a language extension isn't globally unique then check if we have a heuristic | ||
| Linguist::Language.all.each do |language| | ||
| define_method "test_#{language.name}_has_adequate_heuristics" do | ||
| language.extensions.each do |extension| | ||
| languages = Language.find_by_extension(extension) | ||
| next if extension == ".script!" || languages.size == 1 | ||
|
|
||
| assert Heuristics.defined_for?(languages), "No heuristic for #{extension} and #{output_languages(languages)}" | ||
| end | ||
| end | ||
| end | ||
| end | ||