mirror of
https://codeberg.org/gigirassy/riptide
synced 2026-08-30 23:27:39 +00:00
61 lines
2.0 KiB
Markdown
61 lines
2.0 KiB
Markdown
# riptide
|
|
|
|
quick, recursive, sitemap url extractor
|
|
|
|
riptide, with 16 workers, can rip through a 121,000+ link sitemap in less than 6 seconds.
|
|
|
|
```
|
|
nune@nunes-pc ~/sitemaps2> time riptide \
|
|
--sitemap https://www.************.com/sitemap.axd \
|
|
--output ****.txt \
|
|
--scheme https \
|
|
--workers 16 \
|
|
________________________________________________________
|
|
Executed in 5.99 secs fish external
|
|
usr time 528.67 millis 1.14 millis 527.53 millis
|
|
sys time 201.95 millis 0.21 millis 201.74 millis
|
|
```
|
|
|
|
## installation and usage:
|
|
|
|
### install (linux static binary, x86_64)
|
|
|
|
*this assumes /usr/local/bin is part of your path, youll need sha256sum*
|
|
|
|
*concerned about what the script does? check here: https://codeberg.org/gigirassy/riptide/src/branch/main/misc/riptideinstaller.sh*
|
|
|
|
``curl https://riptide.garbageos.com | bash``
|
|
|
|
### install (cargo)
|
|
|
|
``cargo install --git https://codeberg.org/gigirassy/riptide/``
|
|
|
|
|
|
|
|
### usage
|
|
|
|
**linux/bsd**
|
|
|
|
```bash
|
|
riptide \
|
|
--sitemap https://example.com/sitemap.xml \
|
|
--output examplecomurls.txt \
|
|
--scheme https \
|
|
--workers 16 \
|
|
--user-agent "Mozilla/5.0 (X11; Linux i686; rv:151.0) Gecko/20100101 Firefox/151.0"
|
|
```
|
|
|
|
**windows**
|
|
in same directory as riptide.
|
|
|
|
```bash
|
|
.\riptide.exe --sitemap https://example.com/sitemap.xml --output examplecomurls.txt --scheme https --workers 16 --user-agent "Mozilla/5.0 (X11; Linux i686; rv:151.0) Gecko/20100101 Firefox/151.0"
|
|
```
|
|
|
|
for searching for terms in urls after output is completed, i recommend [ripgrep](https://github.com/BurntSushi/ripgrep).
|
|
|
|
by default, riptide identifies itself in user agent, the above example overrides it.
|
|
|
|
## license
|
|
|
|
isc |