Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Feature/Idea Suggestions!! #120
Comments
Metadata extractorExtract webpage metadata like keywords, title, description etc., in order to analyse the site in the future. Image grabberAn image grabber could grab images from the webpages, analyse the image using some algorithm for classification. Randomizer (Identity)Sending multiple requests while using the same IP or header might get our tool to get blocked. To avoid that we can randomise the IP and header for each request. |
|
|
Ok. I will edit my comment and add the details there itself. |
|
I think extracting the title for each webpage will be a good addition to the tree. I am willing to work on this. |
|
@shashwat1002 Great! You can start working on it. |
|
@shashwat1002 Sounds good, let us know if you need assistance. |
|
Hi team, great work with this project. Very impressive. I'm curious, do you plan to add a feature where we can somehow authenticate to an onion service before scanning, as most links/info on many DNMs will be protected with a login system. Keep up the great work! |
|
We don't have plans of implementing such a feature. Would you be willing to work on it? |
|
Hi guys, have you got any plans to add Raw Protocol Info e.g. SSH Key, SSH Fingerprint and Software Banner ? Keep up the great work! |
All suggestions are welcome. Let's talk about what you'd like to see be added to TorBot and we can work on incorporating it into v2. This will be used to track your suggestions.