Skip to main content
added 58 characters in body
Source Link
Stewart
  • 16k
  • 5
  • 49
  • 101

You can use 2 wgets:

  1. wget an HTML page from the website that contains the list of files available for download. Then extract each href=....
  2. Filter those hrefs for patterns that match what you are looking for. This might be something like:
https://download.java.net/java/GA/jdk(*)/*/[0-9]/GPL/openjdk-(*)_linux-x64_bin.tar.gz
  1. Use capture groups (example: ([0-9]+\.[0-9]+)) to extract the version from each URL

  2. Select which version you want to download. This might be the highest version, or it could be the version matching your variable ${VERSION}

  3. wget that url


This is very similar to the uscan tool which is used by debian. uscan will use a file called debian/watch to do exactly what I mentioned above, except it will usually either identify the latest version of a package (and inform developers that one of their packages has an update available), or download it immediately so the developer can import it into debian.

uscan does a bit more than just that, including renaming the downloaded archive to match the name of the package as it appears in debian. That depends on some other proximate files existing, so uscan may not work out-of-the-box for you. It at least suggests that the design of downloading the index, parsing the hrefs, selecting the best one, and wget-ing that works pretty well. uscan is a perl script which isn't too complicated (~4000 sloc with support for lots of stuff you can purge) so you may be able to simply adapt it for your needs.

You can use 2 wgets:

  1. wget an HTML page from the website that contains the list of files available for download. Then extract each href=....
  2. Filter those hrefs for patterns that match what you are looking for. This might be something like:
https://download.java.net/java/GA/jdk(*)/*/[0-9]/GPL/openjdk-(*)_linux-x64_bin.tar.gz
  1. Use capture groups (example: ([0-9]+\.[0-9]+)) to extract the version from each URL

  2. Select which version you want to download. This might be the highest version, or it could be the version matching your variable ${VERSION}

  3. wget that url


This is very similar to the uscan tool which is used by debian. uscan will use a file called debian/watch to do exactly what I mentioned above, except it will usually either identify the latest version of a package (and inform developers that one of their packages has an update available), or download it immediately so the developer can import it into debian.

uscan does a bit more than just that, including renaming the downloaded archive to match the name of the package as it appears in debian. That depends on some other proximate files existing, so uscan may not work out-of-the-box for you. It at least suggests that the design of downloading the index, parsing the hrefs, selecting the best one, and wget-ing that works pretty well. uscan is a perl script which isn't too complicated so you may be able to simply adapt it for your needs.

You can use 2 wgets:

  1. wget an HTML page from the website that contains the list of files available for download. Then extract each href=....
  2. Filter those hrefs for patterns that match what you are looking for. This might be something like:
https://download.java.net/java/GA/jdk(*)/*/[0-9]/GPL/openjdk-(*)_linux-x64_bin.tar.gz
  1. Use capture groups (example: ([0-9]+\.[0-9]+)) to extract the version from each URL

  2. Select which version you want to download. This might be the highest version, or it could be the version matching your variable ${VERSION}

  3. wget that url


This is very similar to the uscan tool which is used by debian. uscan will use a file called debian/watch to do exactly what I mentioned above, except it will usually either identify the latest version of a package (and inform developers that one of their packages has an update available), or download it immediately so the developer can import it into debian.

uscan does a bit more than just that, including renaming the downloaded archive to match the name of the package as it appears in debian. That depends on some other proximate files existing, so uscan may not work out-of-the-box for you. It at least suggests that the design of downloading the index, parsing the hrefs, selecting the best one, and wget-ing that works pretty well. uscan is a perl script which isn't too complicated (~4000 sloc with support for lots of stuff you can purge) so you may be able adapt it for your needs.

Source Link
Stewart
  • 16k
  • 5
  • 49
  • 101

You can use 2 wgets:

  1. wget an HTML page from the website that contains the list of files available for download. Then extract each href=....
  2. Filter those hrefs for patterns that match what you are looking for. This might be something like:
https://download.java.net/java/GA/jdk(*)/*/[0-9]/GPL/openjdk-(*)_linux-x64_bin.tar.gz
  1. Use capture groups (example: ([0-9]+\.[0-9]+)) to extract the version from each URL

  2. Select which version you want to download. This might be the highest version, or it could be the version matching your variable ${VERSION}

  3. wget that url


This is very similar to the uscan tool which is used by debian. uscan will use a file called debian/watch to do exactly what I mentioned above, except it will usually either identify the latest version of a package (and inform developers that one of their packages has an update available), or download it immediately so the developer can import it into debian.

uscan does a bit more than just that, including renaming the downloaded archive to match the name of the package as it appears in debian. That depends on some other proximate files existing, so uscan may not work out-of-the-box for you. It at least suggests that the design of downloading the index, parsing the hrefs, selecting the best one, and wget-ing that works pretty well. uscan is a perl script which isn't too complicated so you may be able to simply adapt it for your needs.