From 232966c4cbdb34b2ad77759c35763270ee45961a Mon Sep 17 00:00:00 2001 From: Rustem Kamalov Date: Thu, 1 Jun 2023 22:33:15 +0300 Subject: [PATCH] Add `gogetcrawl` (#128) * Add `gogetcrawl` --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index bf7d554..6629884 100644 --- a/README.md +++ b/README.md @@ -127,6 +127,7 @@ This list of tools and software is intended to briefly describe some of the most ### Utilities * [ArchiveTools](https://github.com/recrm/ArchiveTools) - Collection of tools to extract and interact with WARC files (Python). +* [Go Get Crawl](https://github.com/karust/gogetcrawl) - Extract web archive data using [Wayback Machine](https://web.archive.org/) and [Common Crawl](https://commoncrawl.org/). *(Stable)* * [gowarcserver](https://github.com/nlnwa/gowarcserver) - [BadgerDB](https://github.com/dgraph-io/badger)-based capture index (CDX) and WARC record server, used to index and serve WARC files (Go). * [har2warc](https://github.com/webrecorder/har2warc) - Convert HTTP Archive (HAR) -> Web Archive (WARC) format (Python). * [httpreserve.info](http://httpreserve.info/) - Service to return the status of a web page or save it to the Internet Archive. Returns JSON via browser or command line via CURL using GET (Golang Package). *(Stable)*