> This friend recommended nvd3.js, presumably because you’re not making real graphs in 2016 unless your graphing library is <something>.js and requires at LEAST one other <something else>.js as a dependency. Everyone looks at you like “what, you DON’T already use <something else>.js? Jeez say goodbye to your Hacker News karma. Just apt-get install npm && npm install bower && bower install-” NO STOP IT THIS ISN’T WHAT TIM BERNERS-LEE WANTED”.
edit: as huckyaus mentioned in a different thread, author did http://swagify.net/ as well. In completely unrelated news, I'm changing my handle to [Tr1Ck$h0t][LEGIT][60x7]$$$C30C0DER$$$, that will make me really popular among the cool kids.
This one got me in stitches, a collection of developer insider jokes in one huge sentence:
AND ANOTHER THING when it’s midnight and your x-axis formatting function doesn’t convert UNIX times into JavaScript date objects properly because there’s no timezone information and I dunno JavaScript was written by some guy in two weeks (yeah I ain’t afraid to call it out what of it) and your binary-search based conversion of sparse timeseries data into uniformly dense timeseries data is causing so many data points to be graphed that it’s slowly crashing Chrome and you’re watching helplessly as your RAM goes up and Chrome won’t close the tab and it just doesn’t seem right that 2016, the year of the Linux Desktop has brought us this situation I mean I thought if you had enough <something>.js libraries this stuff was meant to just scale right up so tha-
The whole article was fun. I loved it. Also to take it a step further, I wouldn't apt-get install npm, I'd first use ruby to install homebrew, then brew install nodejs, then npm install bower, then bower install. Because linux doesn't have enough package managers to manage my package managers to manage packages managed my package managers.
Tho only drawback I see with nvd3.js is that we can't generate png images of the graphs, server-side, for email of PDF export. Is there an nvd3-js-to-png.java ;)?
Note: you can use apt-get install npm and npm to install phantomjs which is not a node.js program but for some reason is on npm, or you can just rent an entire AWS region to compile phantomjs from sources in your preferred linux distribution.
Some people might take issue with it, but the writing for this had me in stitches. I very much agree with the author on graphing libraries - there are a few good simple ones, but as soon as you want anything unusual you have to jump to these big, hard to configure monstrosities. More than once I've just given up and written my own server-side generator.
The writing style works because those jokes, asides, and little mannerisms that are normally excluded from "serious" writing are part of how we think.
When we refine our writing (or speaking) to - supposedly to make it less jumbled and easier to read - we usually end up removing a lot of the little details that made up the original thought. The writing may have much better grammar and organization, but real thought isn't perfectly organized and we end up losing some of the "character" in the original idea. (this may be good or bad, depending on the situation)
Vi Hart has an amazing overview[1] of this concept, in which she discusses the very wise words of the Edmund Snow Carpenter in his[2] book "They Became What They Beheld".
The trouble with knowing what to say and saying it clearly & fully, is that
clear speaking is generally obsolete thinking. Clear statement is like an
art object: it is the afterlife of the process which called it into being.
The process itself is the significant step and, especially at the beginning,
is often incomplete & uncertain.
The best way to write will always depend on what the goals are, but in general the further away from the original thought you get, the more writing and other creative ideas tend to degrade. Incidentally, this tendency of a thought to degrade the further it gets from it's origin is also how plans become policy: [3].
>>The best way to write will always depend on what the goals are, but in general the further away from the original thought you get, the more writing and other creative ideas tend to degrade.<<
So very true...
Open a vein...share yourself...reveal yourself...share what you've learned and let the excitement you felt while discovering it flow...
For most writers, the majority of the time, your true audience will "get" both you and the ideas you're trying to share...
Sometimes you'll expand the size of your potential audience with raw enthusiasm, even when they have no idea what you're talking about...
I think in some cases it is because people are taught that Comma Splicing Is Bad. But imo Ellipse Splicing Is Worse (and should probably be Considered Harmful ;) ). Commas or Em Dashes[0] are better, clearer, and (AFAIK) more grammatically correct.
I was thinking at the same thing today reading some tutorial. You have these serious ass tutorials that are an horror to read. And then you have these gems, these very informal tutorials with jokes and slangs and ... and reading them is a pleasure.
I guess if you're a serious company like Apple, you can't really have that kind of tutorial for iOS dev. But that would really help.
> I guess if you're a serious company like Apple, you can't really have that kind of tutorial for iOS dev.
The fact that this is true is, to me, a slightly sad reflection on our culture. I understand that too much silliness can get in the way of conveying information, but as you point out, so can too much seriousness.
It's obviously a fine balance, and unfortunately it seems safer to err on the side of seriousness when in doubt. It's one of those things that if you sit down and ask any average individual, they will probably prefer a bit of silliness and an informal style in any situation where it's appropriate, yet somehow collectively we seem to have agreed to default to the opposite, and stick with things being quite dry and serious as a norm.
Interesting, but ultimately a little sad. Why is this the case? Why is our culture this way? Why can't things just be a little more fun by default?
I would have put it a little differently. It seems more about image to me than strictly "joking isn't appropriate in business." Apple has an image that is one of sleek, refined beauty. The style of writing used in this article would totally clash with that, causing nothing but confusion and weak branding.
On the other hand, there are brands out there with the exact opposite approach. Cards Against Humanity comes to mind first. Their image is explicitly one of mild carelessness and apparent candidness such that anything written too formally would seem out of place.
But indeed, there do seem to be more of the former brand type than of the latter. Having a neighbor who speaks constantly with a similar tone as the writer of this article, I can say that trying to hold my own in a conversation with him is really tiring. You eventually just run out of witty remarks. Is it possible it's more an issue of skill and time? As in, are there fewer people out there with the skills necessary to make good silly writing than there are people who can write seriously? And does it take them longer (and thus cost even more) to make writing in this style?
I rambled a little, but I think the issue is a combination of people equating seriousness with importance, as well as a general lack of creativity.
I put jokes in my resume. You have to have a bit of restraint, but it can be fun and effective. An oatmeal cookie wouldn't be very good if the whole thing was raisins.
In college, I was a mechanical engineer living with a CompSci major, CompE major, and a CompSci / CompE double-major. I'd occasionally stop them when they were explaining things to me with "Explain like I'm a MechE". Slightly less self-deprecating than "mathematically inclined idiot", but same gist.
Because of this, I was also quite useful as a rubber duck for debugging.
Exactly. I'm guessing the reason is that a funny style implies a more personal and subjective sight on the things it describes, and that means it often distinguishes the important and complicated parts from the trivial ones. Serious texts often tend to threat everything the same and you have to think of the practical impact yourself.
As a not-so-junior dev, I didn't think he was all that funny half the time, and there was enough extraneous verbiage that I didn't bother parsing it all to find out what he was actually doing. I like the humorous style, but IMO this was badly in need of a once (or twice)-over to clean it up.
I agree with you. It must be an youth generation thing, I am in my mid 40s. At times, I find his writing style rather annoying, it feels like one of those TV adverts you see at 1am in the morning.
It is a pity because the content is quite interesting but we are here discussing about his writing style??
Do you have stairs in your house? (i.e. do you browse SomethingAwful, because your style of sarcastic humour is pretty similar, going by your writing & the animated frog on Swagify, and yes yes i know the rules of HN...)
I love this guy's humor. I remember when I was 17 I used to write a column for flipcode.com where I thought I was being pretty funny in appropriate amounts (http://www.flipcode.com/archives/Theory_Practice-Issue_06_Ev... for example) but I like his jokes more!
But hey, I was 17 and also it was me so... I'm biased.
I swear by Highcharts. It's big, and paid for commercial use, but you can chart pretty much anything with it, make it look exactly like you want, and it's relatively easy to get started and make nice interactive charts.
I've used it for school projects and work before, and was amazed by the documentation and API capabilities. Everything is configurable and easy to figure out.
I'm still not sure how the "Free for Non-Commercial" license interacts with something like the GPL - it seems like I'd have to use BSD in my projects instead if I wanted to use this in open-source stuff.
Edit: it's Creative Commons Attribution-NonCommercial 3.0. I wonder if that stops me running ads on a site that uses it.
FWIW in past I just emailed them asking for clarification about whether I needed a commercial license with my specific use case and they responded quickly and straightforwardly. Seems the easiest thing to do is just ask them rather than guessing or trying to interpret how the license is meant to be used.
"The empirical findings suggest that creators and users approach the question of noncommercial use similarly and that overall, online U.S. creators and users are more alike than different in their understanding of noncommercial use. Both creators and users generally consider uses that earn users money or involve online advertising to be commercial"
They're great, and I guess using them wouldn't really matter for this blog post. But they're useful for generating static things. As soon as you generate lots and lots of graphs dynamically, it's easier to offload it to the client. They'll download the JS library once (maybe even from the CDN, not from you) and you just push the data points which are 1/1000th the size of a rendered graph. (they'll also look better, because you can't do subpixel antialiasing without knowing the display)
> If you reload the page you’ll see approximately fifty-bajillion network requests go off as Facebook desperately tries to load all the junk that it needs to display facebook.com.
I like this part. As a developer I've often looked at the network usage of large websites / web applications and it's always surprising to me just how...unoptimized it is as far as network connections go.
I mean Facebook loads decently enough and all I'm just surprised the first load isn't condensed into a small, handful of network calls to save on latency.
> I like this part. As a developer I've often looked at the network usage of large websites / web applications and it's always surprising to me just how...unoptimized it is as far as network connections go.
Have you considered that your opinion on the best way to serve data to a billion users might actually be wrong? That maybe Facebook actually knows what they are doing?
Facebook absolutely hammers client-side system resources. Opening a few Facebook tabs, especially in Chrome, will bring a new fancy computer to its knees, and closing Facebook (either the website or the mobile app) famously substantially extends laptop/phone battery life. Facebook’s site architecture demonstrates staggering lack of respect for their users, even without considering the privacy implications.
Not really surprising, from what I’ve heard from friends about their cowboy-style internal engineering culture, and spaghetti piled on spaghetti internal code organization, where shipping new features ASAP is privileged above all other goals.
When you have endless money to throw engineers and servers at a problem, it’s possible to kinda sorta paper over a lot of horrible broken design. Not sure whether that implies anything one way or another about whether or not they “know what they are doing”.
It sort of depends on your definition of “know”. After all, their motto is “move fast and break things.”
> Have you considered that your opinion on the best way to serve data to a billion users might actually be wrong? That maybe Facebook actually knows what they are doing?
Certainly! But I've also done work where Humvees in the middle of Iraq use satellite internet connectivity with massive latency and the more connections it has to open to get started the slower everything gets.
Perhaps this is an advantage where latency is low so you can load parts of the page faster but I'd love to know some of the more technical reasons behind this!
This is the company that bragged about having 18,000 classes for their iOS app [1] and having to patch the Dalvik Virtual Machine to support new features in their Android app [2]; they are not infallible.
IiRC they aso exploited an Android bug to give themselves more memory. Facebook is woefully buggy and slow that many people are now either using the website or alternative wrapper apps that don't drain their battery immensely.
Given the age of Facebook and what you're describing, I can only assume that Facebook have just chucked fifty-bajillion interns (one after the other) at the source and told them to add more crap.
I mean, isn't that how monolithic apps tend to happen? Someone leaves, new person comes along and adds more to the cruft.
I mean, honestly, that's almost exactly what they said they do with the mobile app and how it contains so many classes / objects that it's really incredibly the app still works.
But I don't know anyone at Facebook so I have no idea how the website is handled.
As someone else have mentioned most of these connections are progressive connection. Another thing to consider is their CDN usage. Facebook uses Akamai as their CDN of choice, most large to medium size telco/ ISP has their own Akamai Cache server at their NOC. So latency is not issue for the big files (images and JavaScript) after the initial reload, assuming they are not cached on the browser already.
So no matter how many connections are made when visiting facebook, it usually loads fairly fast because of aggressive caching and progressive connections. Most of the time the end user shouldn't be able to tell the difference.
The Facebook mobile app on iOS is awful - worse than their mobile website. It takes an age to load, drinks battery life, is massive, doesn't contain the messenger functionality (you need another 60MB app for that).
Don't get me started. The best thing I've ever seen is a site using Google tag manager to load an ad tag manager which loaded a different gtm instance which... Loaded the site's own js libraries.
This horror out of time and space was beaten, shot and stabbed, and the developer responsible for its invention was promoted to CTO and King of the universe.
"I mean Facebook loads decently enough and all I'm just surprised the first load isn't condensed into a small, handful of network calls to save on latency."
Nope. Did you read through how BigPipe works? It's more than a handful; they basically load a whole bunch of mini web pages inside of another. I'm actually impressed they get better performance doing that; I would have thought condensing everything down would net much better performance. It would at least improve latency but I digress.
Yet somehow it's ridiculously fast... like, when I'm waiting for some other site to load, I almost automatically open facebook or hn in another tab, because I know those two sites will load instantly while pretty much everything else on the internet will take a few seconds.
Really? The notifications and the first story show up fairly quickly for me, but everything else usually takes a few seconds. Certainly not terrible, but I wouldn't call it "ridiculously fast" either.
I like to do this sort of web spelunking all the time.. But the writing and humor really make this more enjoyable than it should be! Of course Facebook leaks info to you about your friends - that is the sole attraction for people to use it! Seems like you could turn this thing into a browser extension as well if you felt daring.. Like some sort of FB snooper.
"If you I dunno, didn’t have a lot of friends in high school, you might recognise that as a UNIX time stamp - the time in seconds since midnight, January 1, 1970. "
Great article. And a further reminder why Facebook kinda sucks.
Not sure if you're seriously asking but OP was likely saying it's a great article because of the witty writing. The remark about FB sucking was related to the article's substance.
As someone who is not personally humiliated by my interest in computers/tech/programming, I wasn't really entertained by the constant "oh yea lol it's cause I'm a waste of oxygen that I know that, don't you hate me as much as I hate myself?" Maybe I know too many nerds with actual self esteem issues to find it funny.
It's not humorous because it's funny that he would have low self-esteem; it's humorous due to the irony in the juxtaposition of a demonstration of something so obviously clever and impressive and an over-the-top embodiment of a self-effacing stereotype. Maybe he's putting it on a bit thick, but I think it's well done.
I don't think the OP feels humiliated - if anything, I think what many of us find entertaining is that he recognises the perceptions of nerds/geeks to some folks and is able to laugh about those perceptions / prejudices them while relating to himself; just a different of way of looking at his writing.
Were you equally aggravated by the people who said how much they liked the writing style? Sorry, won't be able to get into a long conversation, I'm heading to a bar.
This style of writing is so entertaining; it's like a funnier stream of conscious of what goes on in my head when hacking things like this together. If OP is the author, please write more.
>So does that mean that the spiky periods are times when people are online the whole time?
I'd be inclined to think so, but I don't work for Facebook, there may be an entirely different reason :)
Edit: or maybe, maaaybe, it's the point of time when people fall asleep over their mobile - they've stopped interacting with the app, it still sends a few keep-alive requests, and then logs itself out.
You should try to add some sort of tracking of when people start to write a message to you. If that's in any way possible. It would be really stalkery if you knew whenever someone started writing a message and perhaps decided not to send the message.
I don't have a Facebook account, but is there really no way to not share your available status to your friends? In Gmail you can simply sign out of Hangouts.
On a side note,
> If you’re wondering why the response starts with “for (;;);”, it’s to, among other things, encourage developers to use a quality JSON decoder, instead of like, y’know, eval().
This is wrong, as I commented on the linked StackOverflow post, perhaps a bit too strongly. But it's really frustrating to see that people have misconceptions because of incorrect answers on StackOverflow.
Any messages you get when your chat is off just go to your Facebook inbox so you can still get messages even with it off you just kinda lose that online live kinda element. I mean this works just like most chat applications work. They basically all say you're online or not too.
You don't even have to sign out of Hangouts - Hangouts has an option to not display "last seen" status which will make you always appear offline (though you can still send and receive messages).
It would be real creepy if someone does the same thing for Whatsapp, you can even predict who's talking to each other much better than Facebook. It's a bit harder to collect data from web.whatsapp.com because it's using Websockets but let me know if someone develop such tool and publish it on Github. :)
Just so you know, at GitLab we will not hesitate to take down content that we think is illegal or has other issues. At this point I think that WhatsSpy is acceptable but we reserve the right to change our mind.
Since it's on a custom domain, I thought that he's using open-source version of Gitlab hosted on his personal server. Do you have control over projects in open-source version of Gitlab?
There's a subset of my Facebook friends, mostly older/family who are slow to reply to messages on Facebook and only use Facebook in 'down time' or whatever but are available by text all day. Since they got smartphones years ago, by text has meant by Whatsapp.
The way I was looking at it was people will get messages on WhatsApp but won't reply immediately. Same on Facebook. But people will also visit Facebook.com when ignoring messages so you still get online status info even if they aren't specifically using the chat. If you could look at WhatsApp and Facebook and combine the data you'd probably get a really accurate overview.
I know I'm in the minority here, but I just couldn't bear the writing style. I'm sure the content is interesting, but this article tries way too hard for my tastes. I had to give up after the first couple of paragraphs.
I did the same thing with the XMPP interface before they scrapped it and it was obviously much easier...also I used the built in graphing that's in Racket to visualize it. Also I made a thing to do desktop notifications whenever someone came online, which is actually kinda useful.
Nice work! I really like the idea that the web allows anyone to programmatically dig into the UI and extract data to do things. A friend and I actually made a whole API to interact with FB chat. You should check it out: https://github.com/Schmavery/facebook-chat-api. I'd really love to see what you can come up with, with some of the stuff we support.
> when it’s midnight and your x-axis formatting function doesn’t convert UNIX times into JavaScript date objects properly because there’s no timezone information and I dunno JavaScript was written by some guy in two weeks (yeah I ain’t afraid to call it out what of it) and your binary-search based conversion of sparse timeseries data into uniformly dense timeseries data is causing so many data points to be graphed that it’s slowly crashing Chrome and you’re watching helplessly as your RAM goes up and Chrome won’t close the tab and it just doesn’t seem right that 2016, the year of the Linux Desktop has brought us this situation I mean I thought if you had enough <something>.js libraries this stuff was meant to just scale right up so tha-
So, did you forget everything you learned about memory management? Or do you think Javascript really doesn't have sound memory management principles? Hell, it's not like you need to retain references to rendered points. Just dequeue them. Browser graphing libraries render to canvas which is just pixels.
You used to be able to do this with the Facebook Query Language (FQL) that Facebook exposed, sending something like this query to the FQL endpoint.
SELECT uid, name, online_presence
FROM user
WHERE online_presence IN ('active', 'idle')
AND uid IN (
SELECT uid2 FROM friend WHERE uid1 = me()
)
Unfortunately, the current version of the Facebook Graph API[0] doesn't have the online_presence field, so this is no longer possible. Maybe the Graph API will be updated in the future to also return the online_presence fields?
What determines whether the app is online? What happens when the user is using the phone but FB is in the background? Does FB get some kind of update when the user is active on device? Or do OP's friend live in the FB app all day long?
I followed his GitHub link at the bottom of the post and see that he's Australian. My preconceived stereotypes of Australians suddenly explains quite a bit of his writing style and humor, and makes the post that much more enjoyable.
Reminds me of the old-school user tracker (whose name escapes me) that would give you a bar graph of your friend's online/offline presence when AOL Instant Messenger was the dominant chat client.
This is awesome! Thank you for sharing the code for this. Overall I would say that this could be very entertaining to watch over multiple sites. Potentially gathering a good profile of your friends over time!
Awesome and hilarious article. Id just like to note `for (;;);` is not to prevent users from using bad JSON parsers like `eval` but prevent older browsers with little to no cross domain policy from loading it with a script tag and doing evil XSS by overriding Array or Object constructors or prototypes to pull that data
No, CSRF (cross-site request forgery) is where a page tricks your browser into making requests to another domain in which you're already authenticated, in order to perform some kind of action. e.g. an img or script with a src "http://example.com/message.php?message=you+are+hax0red&s.... You can sometimes perform similar tricks with self-submitting hidden forms, or XHR. Quite easy to mitigate using nonces and referrer checking.
But.. that's exactly what while(1); and friends in json responses protect you against? someone overriding the Array constructor function and including your JSON resource from a <script src=…> ? So this is, in fact, CSRF?
No, CSRF isn't about pulling scripts etc from another site, it's tricking the browser into making malicious HTTP requests. So, it's not trying to grab facebook .com/someinterestingdata.json, it's trying to trick your browser into performing actions on the target domain by making it perform GET or POST requests such as sending Facebook spam. It doesn't matter what the response is, it's just interested in the action. A while(1) won't do much if it's inside an img tag or hidden iframe rather than a script tag.
Cool hack and awesome, fun writing, but on a more serious note : how is knowing your friends' usage/sleep patterns useful in any way? Could it be used for some dark, machiavélique purpose?
And how about for advertisers? "Get your sleeping pills here" type ads?
Turning off chat doesn't necessarily suggest that is true. While `sticky_pool=atn2c06_chat-proxy&state=active` suggests the case, it's something to explore and see if there is a larger issue at hand.
This is crazy, I had this idea like 1 month ago and I thought I will find some free time to make this happen by the end of this month, I guess I should thank you :)
Nice story by the way :)
I had the idea a month ago too. I even started working on this yesterday but got distracted. Thought I would do it today, but woke up to see this. This is even crazier.
You can use the MQTT Facebook plugin for Bitlbee and get similar online/offline/active information right there in your IRC client without all the screen-scraping faff.
This is getting downvoted, but I really thought D3.js was the most popular library for doing that kind of stuff? I'm a bit outdated I still use gnuplot :/
inb4 facebook resolves this issue by banning anyone who's connected 24/7. (that wouldn't solve the problem either way, btw -- a small group of people could conspire to pull this data at irregular intervals and then share the data with one-another to get a more complete picture while still staying reasonably undetectable if done right.)
I would presumably work for anyone you are chatting with; that’s mostly your friends, but there are edge cases like people who asked to become your friends and haven’t accepted yet, friends of friends in a group chat, with whom you might be able to chat, etc. I haven’t tried though.
A. you get way funnier screen grabs, and honestly simpler controls;
B. you will get love from the security engineers, which I’ve tried, and is an awesome idea.
As it says in the article, this returns a list of the statuses of your friends, he's not querying people individually. He is, however, storing the records individually, but that's a function of his program, not how the API behaves.
Nice with this data I might finally and truly finish my Social Alarm Clock idea and do so in which it truly improves the sound of your alarm clock; one that always makes you smile, laugh, etc.
There's been tons of social alarm clocks(from Justin Bieber to Nestle to Sony to Wakie, etc) since releasing sleep.fm in 2007 (a century ago in Internet years) yet no one has executed on the idea properly.
Social engineering, in the context of information security,
refers to psychological manipulation of people into performing
actions or divulging confidential information.
I don't see how this is social engineering at all.
> This friend recommended nvd3.js, presumably because you’re not making real graphs in 2016 unless your graphing library is <something>.js and requires at LEAST one other <something else>.js as a dependency. Everyone looks at you like “what, you DON’T already use <something else>.js? Jeez say goodbye to your Hacker News karma. Just apt-get install npm && npm install bower && bower install-” NO STOP IT THIS ISN’T WHAT TIM BERNERS-LEE WANTED”.
edit: as huckyaus mentioned in a different thread, author did http://swagify.net/ as well. In completely unrelated news, I'm changing my handle to [Tr1Ck$h0t][LEGIT][60x7]$$$C30C0DER$$$, that will make me really popular among the cool kids.