Depending on your activity….this can be a fun example. Wget can actually be used to crawl an entire website. A number of dubious uses for this come to mind, but from a practical view it can be used for transferring pre-rendered pages, archival and even syncing.
1 | wget -U "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" --execute robots=off --recursive --no-parent --continue --no-clobber http://www.vassox.com |
