Focused crawls are collections of frequently-updated webcrawl data from narrow (as opposed to broad or wide) web crawls, often focused on a single domain or subdomain.
Earthdata Search is a web application developed by NASA EOSDIS to enable data discovery, search, comparison, visualization, and access across EOSDIS' Earth Science data holdings.
@cantzakas created the SQL query necessary to pull metadata in (dataframehq/whale#140) -- we just have to make the greenplum extractor scaffolding. This should just follow the exact same shape as the Postgres extractor.
For the postgresql dep in api/build.gradle, a version greater than 42.2.6 will fail the following test:
marquez.db.ColumnsTest
Test testPgIntervalOrThrow_pgInterval FAILED
org.junit.ComparisonFailure: expected:<...s 0 hours 5 mins 5.0[0] secs"> but was:<...s 0 hours 5 mins 5.0[] secs">
at marquez
Deep Web Extractor (DWX): Deep Web Extractor system is using statistical machine learning models for crawling and data discovery from the Deep Web (i.e., massive and quality portion of World Wide Web) to build knowledge based databases.
It seems the monthly released github action will get triggered everytime a pr is merged. If it is not a new release, the github action will fail.