Hacker News new | past | comments | ask | show | jobs | submit login

This is a pretty cursory look into some other possiblities of refererless traffic. Given an ordinary browsing session or interaction from an end user, what else could be leading to HTTP requests without the referer header?



1) proxies

2) clever antivirus/firewall software

3) htaccess tricks will often drop headers

4) javascript

5) things like amazon silk

6) people spoofing things to make their browser work

7) anonymizer services

8) proxies


9) links followed from https -> http

10) bookmarks


The most peculiar thing is that links off of https://mail.google.com/ essentially mimics the copy & pasted links (i.e. null header.) However, https://plus.google.com, https://twitter.com, and https://facebook.com all present a referer header to the server.

There's probably room here to do some investigation as to what Google is doing to make outbound links from mail.google.com completely drop the referer. (Also, what about other web based mail clients? Yahoo? MSN? Aol? Corporate Outlook?)


The article in question claims that links followed from https facebook do have a referrer header:

>In testing links from Facebook and Twitter over HTTPS the referer is present in most cases.


That's definitely not to spec if the browser is doing it. http://tools.ietf.org/html/rfc2616#section-15.1.3

"Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol."

However, it's possible that Facebook is passing users through an HTTP gateway.


It looks like the intermediate 301's referer is being passed thru (For example, all links on Twitter get wrapped as a t.co link, and that's what shows up on the server). I'd imagine that analytics being mined are intelligent enough to collapse twitter.com and t.co as the same social origin.

Give it a try yourself: https://twitter.com/vikrum5000/status/256898972478763008 Note the protocol.


Do you imagine 1, 6, and 7 being something the mass market would be using? Or, do you suppose that it is all of these little cuts that becomes that nearly 70% of traffic being without a header?


11) There used to be an issue in select browsers (not sure how prevalent it is anymore) that when a user opened a link in a new tab/window, that the document.referrer would not be set correctly.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: