ict.ken.be

Delivering solid user friendly software solutions since the dawn of time.

HTTP Fundamentals - Notes

Categories: Network Notes

by Scott Allen

  • HTTP: HyperText Transfer Protocol
  • URL: Uniform Resource Locator

urlScheme://host:port/urlPath?q=query#fragment

URL Encoding
Safe Characters: a-z A-Z 0-9 $-_.+*'(),

Common MIME Types

  • Type/Subtype
  • application/atom+xml
  • application/json
  • image/gif
  • image/png
  • video/mp4
  • text/xml
  • text/html
  • text/plain

Content Negotiation

html: http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4
date: http://tools.ietf.org/html/rfc822#section-5.1

HTTP Transaction: HTTP Request + HTTP Response

telnet odetocode.com 80
GET /odetocode.jpg HTTP/1.1
Host: www.odetocode.com
enter key twice

HTTP Request Methods
GET, POST, PUT, DELETE, HEAD
Safe versus Unsafe: http://en.wikipedia.org/wiki/Post/Redirect/Get

Request Message
[method] [URL] [version]
[headers]
[body]

GET / HTTP/1.1
Host: server.com
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language:fr-FR
Accept-Encoding: gzip,deflate,sdch
Date: Fri, 9 Aug 2002 21:12:00 GMT
Referer, User-Agent, Accept, Cookie, If-Modified-Since, Accept-Charset

The default q value is 1.0 preference of accept

Response Message
[version] [status] [reason]
[headers]
[body]

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=utf-8
X-headers are reserved for nonstandard headers

301 permanent do not come here again
302 temporarly eg. redirect after sign-in

TCP: Transmision Control Protocol -> Reliable (Transport)
not UDP: User Datagram Protocol -> Unreliable
Opens a socket on IP
IP: Internet Protocol (Network)
Ethernet (Data Link)

private static string GetResource(Uri uri)
{
var hostEntry = Dns.GetHostEntry(host);
var socket = CreateSocket(uri.hostEntry);
SendRequest(socket, uri.Host, uri.PathAndQuery);
return GetResponse(socket);
}

private static Socket CreateSocket(IPHostEntry hostEntry)
{
const int httpPort = 80;
var endPoint = new IPEndPoint(hostEntry.AddressList[0], httpPort);
var socket = new Socket(endPoint.AddressFamily, SocketType.Stream, Protocol.Tcp);
socket.Connect(endPoint);
if (socket.Connected) return socket;
return null;
}

private static void SendRequest(Socket socket, string host, string resource)
{
var requestMessage = String.Format(
"GET {0} HTTP/1.1\r\n" +
"Host: {1}\r\n" +
"\r\n", resource, host
);
var requestBytes = Encoding.ASCII.GetBytes(requestMessage);
socket.Send(requestBytes);
}
private static string GetResponse(Socket socket)
{
int bytes = 0;
byte[] buffer = new byte[256];
var result = new StringBuilder();
do
{
bytes = socket.Receive(buffer);
result.Append(Encoding.ASCII.GetString(buffer, 0, bytes));
}
return result.ToString();
}

http://wireshark.org
Capture Options
Capture Filter: host odetocode.com
3 step TCP Handshake

www.ietf.org/rfs/rfc2616.txt
A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy.

http://en.wikipedia.org/wiki/Slow-start

Default connection close about 5 seconds.
Shared hosts often send 'Connection: close' in header to allow more connections.

HTTP Architecture

http://odata.netflix.com/v2/Catalog/ (https://hidemyass.com/)
REST: www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
Forward Proxy: using a proxy as an access control device (by dropping messages)
Reverse Proxy: eg. loadbalancer
Fiddler installs itself as a local 127.0.0.1:8888 proxy in your browser

Caching
Chrome: about:cache

Cache It: HTTP/1.1 200 OK GET
Don't cache: PUT, POST, DELETE

Cache-Control: private, max-age=0
Expires: ... should no longer be used
Pragma: ... should no longer be used

Cache Control

  • public: A response for everyone
  • private: A response for a single user
  • no-cache, must-revalidate: Don't cache the response
  • no-store: You never saw this response
Response.Cache.SetCacheability(HttpCacheability.Public);
Response.Cache.SetExpires(DateTime.Now.AddSeconds(60));

If-Modified-Since: Wed, 14 Sep 2011 19:02:18 GMT

Etag: "07f2a22ffd9cc1:0" hash of the resource
If-None-Match: "07f2a22ffd9cc1:0"

HTTP Security
Cookies and Authentication
http://tools.ietf.org/html/rfc6265
About 4kb max

Saving State
ViewState <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/foo" />
Store in database
SessionState Store
Cookies

Set-Cookie: ASP.NET_SessionId=foo; path=/; HttpOnly

domain=.searchengine.com (include subdomains)
expires=Monday, 09-July-2012 21:12:00 GMT

Security

Basic Authentication
WWW-Authenticate: Basic realm="localhost"
Authorization: Basic base64encodedusernameandpassword

Digest Authentication
WWW-Authenticate: Digest realm="localhost", qop="auth,auth-int", nonce="...", opaque="..."

Windows Authentication
WWW-Authenticate: Negotiate
NTLM and Kerberos

Forms Authentication
Location: /login?ReturnUrl=/account

OpenID: Google, Yahoo, myOpenId, StackExchange, facebook

HTTPS
Adds a SSL/TLS layer inbetween application and transport layer
Certificate with Public/Private key
Everything except the hostname