From c52695c581865d85d5a8b5482074fc5c0c94bd8b Mon Sep 17 00:00:00 2001 From: gigirassy Date: Sun, 23 Aug 2026 20:55:14 +0200 Subject: [PATCH] Update README.md --- README.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ca258bb..00abc9e 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,19 @@ # blogfetch -tiny crystal web crawler to crawl a blogspot blog and all other blogspot blogs it links recursively until a set amount of urls (5000) is reached \ No newline at end of file +tiny crystal web crawler to crawl a blogspot blog and all other blogspot blogs it links recursively until a set amount of urls (5000) is reached + +this is especially helpful for archiveteam. + +## how to +### ...build +1. clone this repo, cd to it. +2. (linux) ```docker run --rm -it -v .:/tmp/blogfetch 84codes/crystal:latest-alpine build /tmp/blogfetch/blogfetch.cr --release --static --no-debug -o /tmp/blogfetch/blogfetch && sudo chown 1000:1000 blogfetch && strip blogfetch``` will init it all and chown the resulting (enter your pass when prompted) to your user. +2. (other) install crystal from your package manager, then ``crystal build blogfetch.cr --release --static --no-debug -o blogfetch``, strip it (optional), then copy to anywhere in your path. +### ...use +1. find any blogspot blog with a blogroll. +2. affix it to the command, like ``blogfetch [url of that blog here]``. +3. affix ``> blogfetchoutput.txt`` to put output in a file. +4. wait for it to be done crawling, then transfer your file to the place of your choice. + +## license +public domain when legally possible in your jurisdiction, otherwise 0BSD. \ No newline at end of file