Monday, April 12, 2010

WGET - Authentication

How to access a page using wget that requires authentication? wget is well equipped to handle multiple authentication scenario's.

HTTP Basic Athentication: To download a page that requires HTTP basic authentication use the following mechanism:
wget https://myUserName:myPassword@www.myserver.com/mypage.html
wget http://myUserName:myPassword@www.myserver.com/mypage.html

Form Post: To download a page protected by login built on form post use the following:

wget --post-data 'user=myUserName&password=myPassword' http://www.myserver/mypage.html

Form Post with multiple pages:If you need to navigate through multiple pages after authentication to get to your page, you can save cookies on form post for authentication and reuse the cookies file to access the page you want:

wget --post-data 'user=myUserName&password=myPassword' --cookies=on --keep-session-cookies --save-cookies=myCookies.txt http://www.myserver/auth

wget --cookies=on --load-cookies=myCookies.txt --keep-session-cookies --savecookies=myCookies.txt http://www.myserver/mypage.html