Tip: Generating Directory Listings with wget

Posted by Alexander Todorov on Thu 21 February 2013

Today I was looking to generate a list of all files under remote site directory, including sub directories. I found no built-in options for this in wget. This is how I did it:

wget http://example.com/dir/ --spider -r -np 2>&1 | grep http:// | tr -s ' ' | cut -f3 -d' '

I managed to retrieve 12212 entries from the URL I was exploring.

tags: tips



Comments !