riptide

quick, recursive, sitemap url extractor

installation and usage:

git clone https://codeberg.org/gigirassy/riptide/ && cd riptide && cargo install .

riptide \
                               --sitemap https://example.com/sitemap.xml \
                               --output examplecomurls.txt \
                               --scheme https \
                               --workers 16 \
                               --user-agent "Mozilla/5.0 (X11; Linux i686; rv:151.0) Gecko/20100101 Firefox/151.0"

by default, riptide identifies itself in user agent, the above example overrides it.

license

isc