Downloading a website using wget with auth and ssl TEST

  • Posted by Adam Niedzwiedzki on 23 December 2014, 12:55 pm

First off issue the login and store the session cookie.

 wget --no-check-certificate \
--save-cookies cookies.txt \
--keep-session-cookies \
--post-data 'username=user&password=pass&remember=1&Submit=Login' \
https://some.web.site.com

Then we request the page we wish to download and follow.

 wget \
 --load-cookies cookies.txt \
 --keep-session-cookies \
 --save-cookies cookies.txt \
 --reject *index.php*,*whoson.php* \
 --no-check-certificate \
 --recursive \
 --no-clobber \
 --page-requisites \
 --html-extension \
 --convert-links \
 --restrict-file-names=windows \
 --domains some.web.site.com \
 --no-parent \
https://some.web.site.com

Please Register.


If you wish to add comments.
Cheers
Adam