diff --git a/Internet-Tools.md b/Internet-Tools.md index efd0f405d..2bf66a76c 100644 --- a/Internet-Tools.md +++ b/Internet-Tools.md @@ -685,3 +685,88 @@ * [Pagetual](https://hoothin.github.io/PagetualGuide/en/) - Merge Sites "Next" Pages * [Mouseover Popup Image Viewer](https://greasyfork.org/en/scripts/394820-mouseover-popup-image-viewer) - Shows Images & Videos Behind Links & Thumbnails * [DisableAMP](https://github.com/AdguardTeam/DisableAMP) - Disable Google AMP Links + +*** + +# ► Archiving + +## ▷ Archive Services + +* ⭐ **[Archive.org](https://archive.org/)** - Internet Archive +* ⭐ **[Wayback Machine](https://web.archive.org/)** or **[Archive.is](https://archive.is/)** / [.li](https://archive.li/) / [.ph](https://archive.ph/) / [.vn](https://archive.vn/) / [.fo](https://archive.fo/) / [.md](https://archive.md/) - Archive Web Pages +* ⭐ **Wayback Machine Tools** - [Downloader](https://github.com/jsvine/waybackpack) / [Browser Extension](https://github.com/internetarchive/wayback-machine-webextension), [2](https://vegetableman.github.io/vandal/) / [Script](https://github.com/overcast07/wayback-machine-spn-scripts) / [Auto Load](https://gitlab.com/gkrishnaks/WaybackEverywhere-Firefox) +* ⭐ **[Web Archives](https://github.com/dessant/web-archives)** or [Resurrect Pages Fork](https://github.com/Albirew/resurrect-pages-isup-edition) - Browser Extensions +* ⭐ **[CachedView](https://cachedview.nl/)** or [Quick Cache](https://cybdetective.com/quickcacheandarhivesearch.html) - Aggregate Cache Results +* [ArchiveTeam](https://wiki.archiveteam.org/index.php/Main_Page) - Archive Projects +* [Perma.cc](https://perma.cc/) - Create Permalinks + +## ▷ Web Archiving Tools + +* 🌐 **[Awesome Web Archiving](https://github.com/iipc/awesome-web-archiving)** - Web Archiving Tools +* 🌐 **[Webrecorder](https://webrecorder.net/)** - Open-source Archiving Tools +* ⭐ **[ArchiveBox](https://archivebox.io)** - Self-hosted Web Archiving +* ⭐ **[MarkDownload](https://github.com/deathau/markdownload)** - Download Web Pages as Markdown Files +* ⭐ **[HTTrack](https://www.httrack.com/)** / [Guide](https://rentry.co/cloneasite) - Website Downloader +* ⭐ **[datahoarder-website-to-markdown](https://github.com/evilsh3ll/datahoarder-website-to-markdown)** - Index to Markdown Tool +* [WAIL](https://matkelly.com/wail) / [GitHub](https://github.com/machawk1/wail) - GUI For Archiving Tools +* [ReplayWeb.page](https://replayweb.page/) - View Web Archive Files +* [ArchiveWeb.page](https://archiveweb.page/) - Browser Extension +* [WikiTeam](https://github.com/WikiTeam/wikiteam) - Archive Wikis +* [Wayback](https://github.com/wabarc/wayback) - Web Archiving Tool +* [DownloadNet](https://github.com/dosyago/DownloadNet) or [Kiwix](https://kiwix.org/en/) / [Wiki DL Guide](https://practicalbetterments.com/download-all-of-wikipedia-on-your-phone/) - Offline Website Readers +* [Wget2](https://gitlab.com/gnuwget/wget2) / [Commands](https://www.whatismybrowser.com/developers/tools/wget-wizard/), [SuckIT](https://github.com/skallwar/suckit), [Cyotek WebCopy](https://www.cyotek.com/cyotek-webcopy) or [Website Downloader](https://github.com/AhmadIbrahiim/Website-downloader) - Website Downloaders +* [Archivematica](https://www.archivematica.org/) - Digital Preservation System +* [wallabag](https://wallabag.org/) - Save Articles +* [CopySite](https://xdan.ru/copysite/) - Copy Websites +* [Scoop](https://github.com/harvard-lil/scoop) - Capture Engine + +## ▷ Web Scraping / Crawling + +* 🌐 **[Awesome Web Scraping](https://github.com/lorien/awesome-web-scraping)** - Web Scraping Tools +* 🌐 **[Awesome-crawler](https://github.com/BruceDone/awesome-crawler)** - Crawling Resources +* ⭐ **[Instant Data Scraper](https://chromewebstore.google.com/detail/instant-data-scraper/ofaokhiedipichpaobibbnahnkdoiiah)** - Browser Extension +* [Heritrix](https://heritrix.readthedocs.io/) / [GitHub](https://github.com/internetarchive/heritrix3) - Internet Archive's Web Crawler +* [80legs](https://80legs.com/) - Cloud-Based +* [Crawly](https://crawly.diffbot.com/) - Online Scraper +* [web.scraper.workers.dev](https://web.scraper.workers.dev/) - Web Scraper +* [grab-site](https://github.com/ArchiveTeam/grab-site) - ArchiveTeam Web Crawler +* [brozzler](https://github.com/internetarchive/brozzler) - Web Crawler + +*** + +# ► Open Source Intelligence + +* 🌐 **[Awesome OSINT](https://github.com/jivoi/awesome-osint)** - Awesome OSINT +* ↪️ **[OSINT Collections](https://www.reddit.com/r/FREEMEDIAHECKYEAH/wiki/storage#wiki_osint_collections)** +* ⭐ **[IntelTechniques](https://inteltechniques.com/index.html)** or [OSINT Techniques](https://www.osinttechniques.com/) - OSINT Resources +* ⭐ **[Analyst Research Tools](https://analystresearchtools.com/)** - Tools and Resources +* [Osintracker](https://www.osintracker.com/) - Track Your Investigations +* [Cyber Detective](https://cybdetective.com/) / [GitHub](https://github.com/cipher387) - Tools, Techniques, and Projects +* [Project Owl](https://discord.com/invite/projectowl) - OSINT Community +* [OSINT Dojo](https://www.osintdojo.com/resources/) - OSINT Guides +* [Hackers Arise OSINT](https://www.hackers-arise.com/osint) - OSINT Guides +* [What is OSINT?](https://www.recordedfuture.com/blog/open-source-intelligence-definition) - Article +* [The Key to Unlocking the Web’s Secrets](https://www.einvestigator.com/open-source-intelligence-tools/) - Article +* [Toddington](https://www.toddington.com/resources/free-osint-resources-open-source-intelligence-search-tools-research-tools-online-investigation/) - Search OSINT Resources +* [Malfrat's OSINT Map](https://map.malfrats.industries/) - Interactive Map +* [OH SHINT!](https://ohshint.gitbook.io/oh-shint-its-a-blog/) - OSINT Blog + +## ▷ OSINT Collections + +* [Social Media](https://start.me/p/4K0DXg/social-media) - Social OSINT Tools +* [Ph055a's OSINT Collection](https://github.com/Ph055a/OSINT_Collection) - OSINT Investigation Tools +* [Bellingcat](https://docs.google.com/spreadsheets/d/18rtqh8EG2q1xBo2cLNyhIDuK9jrPGwYr9DI2UncoqJQ/) - Online Investigation Toolkit +* [Reuser](https://rr.reuser.biz/) - OSINT Resource Discovery Toolkit +* [GEOINT](https://start.me/p/W1kDAj/geoint) - Geographic OSINT Tools +* [Cyber Detective's OSINT Collection](https://github.com/cipher387/osint_stuff_tool_collection) - General Index +* [Nixintel's OSINT Resource List](https://start.me/p/rx6Qj8/nixintel-s-osint-resource-list) - General Index +* [Technisette's OSINT Tools](https://start.me/p/wMdQMQ/tools) - General Index +* [OSINT](https://start.me/p/ZME8nR/osint) - General Index +* [UK-OSINT](https://www.uk-osint.net/index.html) - General Index +* [AsINT_Collection](https://start.me/p/b5Aow7/asint_collection) - General Index +* [OSINT4ALL](https://start.me/p/L1rEYQ/osint4all) - General Index +* [OSINT All](https://start.me/p/0PwOGl/osint-all) - General Index +* [OSINT Resources](https://start.me/p/1kAP0b/osint-resources) - General Index +* [OSINT Essentials](https://www.osintessentials.com/) - General Index +* [OSINTgeek](https://osintgeek.de/tools) - General Index +* [MIDASearch](https://midasearch.org/) - General Index \ No newline at end of file diff --git a/MISCGuide.md b/MISCGuide.md index b66ba7689..a0cdc2c76 100644 --- a/MISCGuide.md +++ b/MISCGuide.md @@ -80,25 +80,6 @@ *** -## ▷ Open Source Intelligence - -* 🌐 **[Awesome OSINT](https://github.com/jivoi/awesome-osint)** - Awesome OSINT -* ↪️ **[OSINT Collections](https://www.reddit.com/r/FREEMEDIAHECKYEAH/wiki/storage#wiki_osint_collections)** -* ⭐ **[IntelTechniques](https://inteltechniques.com/index.html)** or [OSINT Techniques](https://www.osinttechniques.com/) - OSINT Resources -* ⭐ **[Analyst Research Tools](https://analystresearchtools.com/)** - Tools and Resources -* [Osintracker](https://www.osintracker.com/) - Track Your Investigations -* [Cyber Detective](https://cybdetective.com/) / [GitHub](https://github.com/cipher387) - Tools, Techniques, and Projects -* [Project Owl](https://discord.com/invite/projectowl) - OSINT Community -* [OSINT Dojo](https://www.osintdojo.com/resources/) - OSINT Guides -* [Hackers Arise OSINT](https://www.hackers-arise.com/osint) - OSINT Guides -* [What is OSINT?](https://www.recordedfuture.com/blog/open-source-intelligence-definition) - Article -* [The Key to Unlocking the Web’s Secrets](https://www.einvestigator.com/open-source-intelligence-tools/) - Article -* [Toddington](https://www.toddington.com/resources/free-osint-resources-open-source-intelligence-search-tools-research-tools-online-investigation/) - Search OSINT Resources -* [Malfrat's OSINT Map](https://map.malfrats.industries/) - Interactive Map -* [OH SHINT!](https://ohshint.gitbook.io/oh-shint-its-a-blog/) - OSINT Blog - -*** - # ► Free Stuff * ⭐ **[Give & Away](https://discord.gg/j9fvrxedkX)** - Giveaway Discord Server diff --git a/STORAGE.md b/STORAGE.md index c54347b09..eddd9a6da 100644 --- a/STORAGE.md +++ b/STORAGE.md @@ -115,52 +115,6 @@ *** -## Archiving - -### Archive Services - -* ⭐ **[Archive.org](https://archive.org/)** - Internet Archive -* ⭐ **[Wayback Machine](https://web.archive.org/)** or **[Archive.is](https://archive.is/)** / [.li](https://archive.li/) / [.ph](https://archive.ph/) / [.vn](https://archive.vn/) / [.fo](https://archive.fo/) / [.md](https://archive.md/) - Archive Web Pages -* ⭐ **Wayback Machine Tools** - [Downloader](https://github.com/jsvine/waybackpack) / [Browser Extension](https://github.com/internetarchive/wayback-machine-webextension), [2](https://vegetableman.github.io/vandal/) / [Script](https://github.com/overcast07/wayback-machine-spn-scripts) / [Auto Load](https://gitlab.com/gkrishnaks/WaybackEverywhere-Firefox) -* ⭐ **[Web Archives](https://github.com/dessant/web-archives)** or [Resurrect Pages Fork](https://github.com/Albirew/resurrect-pages-isup-edition) - Browser Extensions -* ⭐ **[CachedView](https://cachedview.nl/)** or [Quick Cache](https://cybdetective.com/quickcacheandarhivesearch.html) - Aggregate Cache Results -* [ArchiveTeam](https://wiki.archiveteam.org/index.php/Main_Page) - Archive Projects -* [Perma.cc](https://perma.cc/) - Create Permalinks - -### Web Archiving Tools - -* 🌐 **[Awesome Web Archiving](https://github.com/iipc/awesome-web-archiving)** - Web Archiving Tools -* 🌐 **[Webrecorder](https://webrecorder.net/)** - Open-source Archiving Tools -* ⭐ **[ArchiveBox](https://archivebox.io)** - Self-hosted Web Archiving -* ⭐ **[MarkDownload](https://github.com/deathau/markdownload)** - Download Web Pages as Markdown Files -* ⭐ **[HTTrack](https://www.httrack.com/)** / [Guide](https://rentry.co/cloneasite) - Website Downloader -* ⭐ **[datahoarder-website-to-markdown](https://github.com/evilsh3ll/datahoarder-website-to-markdown)** - Index to Markdown Tool -* [WAIL](https://matkelly.com/wail) / [GitHub](https://github.com/machawk1/wail) - GUI For Archiving Tools -* [ReplayWeb.page](https://replayweb.page/) - View Web Archive Files -* [ArchiveWeb.page](https://archiveweb.page/) - Browser Extension -* [WikiTeam](https://github.com/WikiTeam/wikiteam) - Archive Wikis -* [Wayback](https://github.com/wabarc/wayback) - Web Archiving Tool -* [DownloadNet](https://github.com/dosyago/DownloadNet) or [Kiwix](https://kiwix.org/en/) / [Wiki DL Guide](https://practicalbetterments.com/download-all-of-wikipedia-on-your-phone/) - Offline Website Readers -* [Wget2](https://gitlab.com/gnuwget/wget2) / [Commands](https://www.whatismybrowser.com/developers/tools/wget-wizard/), [SuckIT](https://github.com/skallwar/suckit), [Cyotek WebCopy](https://www.cyotek.com/cyotek-webcopy) or [Website Downloader](https://github.com/AhmadIbrahiim/Website-downloader) - Website Downloaders -* [Archivematica](https://www.archivematica.org/) - Digital Preservation System -* [wallabag](https://wallabag.org/) - Save Articles -* [CopySite](https://xdan.ru/copysite/) - Copy Websites -* [Scoop](https://github.com/harvard-lil/scoop) - Capture Engine - -### Web Scraping / Crawling - -* 🌐 **[Awesome Web Scraping](https://github.com/lorien/awesome-web-scraping)** - Web Scraping Tools -* 🌐 **[Awesome-crawler](https://github.com/BruceDone/awesome-crawler)** - Crawling Resources -* ⭐ **[Instant Data Scraper](https://chromewebstore.google.com/detail/instant-data-scraper/ofaokhiedipichpaobibbnahnkdoiiah)** - Browser Extension -* [Heritrix](https://heritrix.readthedocs.io/) / [GitHub](https://github.com/internetarchive/heritrix3) - Internet Archive's Web Crawler -* [80legs](https://80legs.com/) - Cloud-Based -* [Crawly](https://crawly.diffbot.com/) - Online Scraper -* [web.scraper.workers.dev](https://web.scraper.workers.dev/) - Web Scraper -* [grab-site](https://github.com/ArchiveTeam/grab-site) - ArchiveTeam Web Crawler -* [brozzler](https://github.com/internetarchive/brozzler) - Web Crawler - -*** - ## Browser eBook Readers * ⭐ **[Reader View](https://webextension.org/listing/chrome-reader-view.html)**, [2](https://mybrowseraddon.com/reader-view.html) @@ -785,28 +739,6 @@ *** -## OSINT Collections - -* [Social Media](https://start.me/p/4K0DXg/social-media) - Social OSINT Tools -* [Ph055a's OSINT Collection](https://github.com/Ph055a/OSINT_Collection) - OSINT Investigation Tools -* [Bellingcat](https://docs.google.com/spreadsheets/d/18rtqh8EG2q1xBo2cLNyhIDuK9jrPGwYr9DI2UncoqJQ/) - Online Investigation Toolkit -* [Reuser](https://rr.reuser.biz/) - OSINT Resource Discovery Toolkit -* [GEOINT](https://start.me/p/W1kDAj/geoint) - Geographic OSINT Tools -* [Cyber Detective's OSINT Collection](https://github.com/cipher387/osint_stuff_tool_collection) - General Index -* [Nixintel's OSINT Resource List](https://start.me/p/rx6Qj8/nixintel-s-osint-resource-list) - General Index -* [Technisette's OSINT Tools](https://start.me/p/wMdQMQ/tools) - General Index -* [OSINT](https://start.me/p/ZME8nR/osint) - General Index -* [UK-OSINT](https://www.uk-osint.net/index.html) - General Index -* [AsINT_Collection](https://start.me/p/b5Aow7/asint_collection) - General Index -* [OSINT4ALL](https://start.me/p/L1rEYQ/osint4all) - General Index -* [OSINT All](https://start.me/p/0PwOGl/osint-all) - General Index -* [OSINT Resources](https://start.me/p/1kAP0b/osint-resources) - General Index -* [OSINT Essentials](https://www.osintessentials.com/) - General Index -* [OSINTgeek](https://osintgeek.de/tools) - General Index -* [MIDASearch](https://midasearch.org/) - General Index - -*** - ## Piracy Site Proxies * ⭐ **[Unblockit](https://unblockit.asia/)** / [/r/Unblockit](https://www.reddit.com/r/Unblockit/)