sallust/awesome-web-archiving
1
0
Fork 0
mirror of https://github.com/iipc/awesome-web-archiving.git synced 2026-05-26 06:17:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add bagnabit2warc utility (#172)

Browse Source
This commit is contained in:
Alex Osborne
2026-03-17 23:02:34 +09:00
committed by GitHub
parent 6a9d393783
commit a19a9466ee
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1
View File
@@ -138,6 +138,7 @@ This list of tools and software is intended to briefly describe some of the most
### Utilities
* [ArchiveTools](https://github.com/recrm/ArchiveTools) - Collection of tools to extract and interact with WARC files (Python).
* [bagnabit2warc](https://github.com/internetarchive/bagnabit2warc) - Convert a [bag-nabit](https://github.com/harvard-lil/bag-nabit) dataset stored in a ZIP into a full-content WARC.
* [cdx-toolkit](https://pypi.org/project/cdx-toolkit/) - Library and CLI to consult cdx indexes and create WARC extractions of subsets. Abstracts away Common Crawl's unusual crawl structure. *(Stable)*
* [duckdb-web-archive-cdx](https://github.com/midwork-finds-jobs/duckdb-web-archive) - DuckDB extension to query the Internet Archive and CommonCrawl CDX APIs directly from SQL. *(In Development)*
* [Go Get Crawl](https://github.com/karust/gogetcrawl) - Extract web archive data using <!--lint ignore double-link-->[Wayback Machine](https://web.archive.org/) and [Common Crawl](https://commoncrawl.org/). *(Stable)*