If you have used an HttpWebRequest to grab a page you’ll find that it is reasonably easy to do – but if you use it to perform an automated login to a website, there are a couple of things you may trip up on.
Proxy says no
If you get an HTTP 407 telling you that your proxy says no, you’ll need to add a proxy to the request. In most cases, you can do this using default values. Alternatively, you’ll need to set up credentials on your proxy for your application.
<system.net> <defaultProxy useDefaultCredentials="true"/> </system.net>
Where are my cookies?
When you perform a login, the request – response flow is normally like this.
- POST login form
- HTTP 302 response
- GET location from 302 response
- HTTP 200 response
When automating a login, this causes a problem, because you probably want to grab hold of the cookies sent in step 2 (the first response) – but you’ll find that the response stream is the one from step 4 (the second response).
You can stop the request from automatically following the redirection and get hold of the first response (step 2) by switching off automatic redirections.
httpWebRequest.AllowAutoRedirect = false;
Keeping hold of cookies
Once you get the response, you’ll need to squirrel the cookies away in a cookie container and then add that cookie container to the subsequent requests, so your login cookies get passed to the server.
Here is a full example where the Login method returns the cookie container to be used on subsequent requests.
I have written this as a mega-method for the purposes of the example – you could easily chop this up to make it re-usable and more readable.
protected static CookieContainer Login() { string userName = "my.test.user.name"; string password = "my.test.password"; ASCIIEncoding encoding = new ASCIIEncoding(); string postData = "Username=" + userName + "&Password=" + password; byte[] postDataBytes = encoding.GetBytes(postData); HttpWebRequest httpWebRequest = (HttpWebRequest) WebRequest.Create("http://localhost/Login/"); httpWebRequest.Method = "POST"; httpWebRequest.ContentType = "application/x-www-form-urlencoded"; httpWebRequest.ContentLength = postDataBytes.Length; httpWebRequest.AllowAutoRedirect = false; using (var stream = httpWebRequest.GetRequestStream()) { stream.Write(postDataBytes, 0, postDataBytes.Length); stream.Close(); } var cookieContainer = new CookieContainer(); using (var httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse()) { using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream())) { foreach (Cookie cookie in httpWebResponse.Cookies) { cookieContainer.Add(cookie); } } } return cookieContainer; }