It's hard to overstate the damage this is causing for merchants that prominently display FB like on their pages - lots of big ugly blank spaces / errors out there.
Realize this is temporary but shows why if you install any external vendor tool as a part of your site make contingency plans that switch on errors.
You're telling me--of course this has to happen the day before client review of a major Facebook Connect site whose only authentication is via Facebook Accounts. We had discussed potentially having two types of accounts (Facebook- and email-based), but having two parallel authentication methods just seemed clunky. I guess this is the price we pay for trying to keep things simple...
Seeing as Facebook login lets you easily collect email addresses these days you could always have a page that says, facebook is down click here to get an email confirmation link to login or something similar.
Unfortunately its difficult to avoid relying on other businesses for crucial systems.
Example: Payment Processing, try building a business, especially an online one, without depending on Paypal, or at least Mastercard/Visa.
Sure you can reduce it by say not having FB Connect as the only log-on option but the simple fact is that you will almost always have to depend on other businesses, the best you can do is diversify and hope for the best.
Exactly...people made this same argument a few years back when Amazon S3 went down for a few hours. Just have a few different options, and the chance that they will all be down at the same time is pretty slim.
Yes and No. If you have a few options than your entire business won't come grinding to a halt but if half of your customers choose to use FB Connect as their login option than they likely don't care that FB's down, or that people with email logins can still access your site. To 50% of your user-base your site has failed and so in that sense I would say that you still rely on FB for your business.
I use Facebook (sometimes), played FarmVille and Farm Town up until this January, and I STILL didn't get it right off. Guess pulling out of those games was a good idea after all. :)
The first 10 times I read this comment, I thought it was a pun on facebook's color scheme, until I realized that it's cornflower blue, not cauliflower blue.
I came here to post this same thing, but I just started thinking about it. When was the last time FB went down like this and took a bunch of core functionality down with it? Has it ever happened? And how long are they down...30 mins? Is that 30 mins of downtime every few years really more damaging than that the (potential) value that integration with the graph API or FB Connect can bring? Yes, you look like an idiot to a few users, but maybe you have 100x as many users as you'd have otherwise. I think the real reason not to build your business on the FB API (or Twitter or whoever) is strategic, not because of potential downtime.
You're right, I should have phrased it "And that, my friends, is one of the many reasons its risky to build your entire business on top of FB Graph API."
Just went through a nightmare building a product entirely on top of fb graph. FB fails, a lot. And not just in terms of downtime, in terms of inconsistently implemented technology, poor documentation, lack of transparency, inconsistent policies etc.
Not saying don't use FB ever, just that it's risky.
Oh, don't get me wrong. I won't do FB apps for clients anymore, despite the high demand and great pay. It's just way too stressful to have the clients always angry at me because of the platform's shortcomings.
I fully agree with you. If your app is heavily social it would be ridiculous not to leverage the worlds largest social network, even with very occasional down-time.
Their SSL site still continues to simply not work:
www.facebook.com uses an invalid security certificate.
The certificate is only valid for the following names:
a248.e.akamai.net , *.akamaihd.net
This is the biggest problem with the SSL PKI. It is partially responsible for the lackluster deployment of SSL. We need to switch to a TOFU model supplemented by a more nimble version of the current PKI for first connect acceptance.
You would not believe how often I run into problems just trying to use SSL on majorsites. If you can't trust a large organizations like facebook and akamai to mess up SSL support during problems, you simply will never get it deployed well on all the rest of the sites.
SSL is simply a poor user experience. Twitter's SSL site often goes down more often than the main site too. SSL users get a second class experience and that is the most damming thing I can think of for a technology where deployment is so critical.
Everyone should calm down and realize that Facebook being down for a few hours is not going to kill anyone's business. It may cause trouble and disruption, but not anything more intense than a daily commute in Los Angeles.
That being said, this makes me question Facebook's ability to deliver on the "Social Graph" promise. If they truly want to build a completely social internet, this kind of downtime can't happen.
> "If they truly want to build a completely social internet, this kind of downtime can't happen."
Why not?
Even commerce sites such as Amazon or Ebay have downtime. Studies demonstrate that unless you're selling a perfectly commoditized generic product, obtainable instantly from multiple sources, downtime doesn't materially impact revenue.
Users seem to manage to "queue up" their intended actions and do them when a site is back. And it's not like Facebook's users can gravitate en masse to anywhere else.
The founding principle of the internet is not "5 nines" uptime for all components, but resiliency. Switch off your mail server for a few hours; you'll still get all your mail when you turn it back on.
"Kids these days" build apps and APIs (and sites that rely on APIs) that are much more brittle, because today's developers are spoiled into thinking communication conduits are reliable. They're not.
The fabric fails, so your site, your apps, your protocols, should fail gracefully -- and recover gracefully later.
As an aside I notice that 'Facebook' is not a trending topic on Twitter. Seems the term must be blacklisted in some way along with all the swears and other undesirable terms.
There are more principled ways to do that than a blacklist, though, like looking at current frequency compared to typical frequency. That way even something commonly mentioned could still genuinely be trending if it gets mentioned much more than usual on a particular day.
(It's possible there are scale-related reasons that make this infeasible, though.)
Just not tracking the frequency of the most common words that will never be in the trending topics is probably a big performance win. I'm sure there is an additional blacklist that includes stuff like "and" and "the".
In researching web performance DNS configurations issues are common problem. It is usually related to www.foo.com and foo.com being configured differently. This seems to be at a high level the issue with facebook.com. if you go to "facebook.com" the DNS will resolve correctly, but it will redirect you to www.facebook.com. I guess they could stop the redirection and fix the problem immediately, or wait till the sorry.ak.facebook.com.edgesuite.net is fixed.
;; ANSWER SECTION:
www.facebook.com. 0 IN CNAME sorry.ak.facebook.com.edgesuite.net.
sorry.ak.facebook.com.edgesuite.net. 0 IN CNAME a1030.g.akamai.net.
a1030.g.akamai.net. 4 IN A 63.84.59.59
a1030.g.akamai.net. 4 IN A 63.84.59.10
When I heard FB was down I went over to twitter to see how many tweets this was generating, I was surprised to see the first refresh of "XXXX more tweets since you started searching."
I decided to start keeping track and I made a graph of the difference between refreshes. You can see my graph here. It seems like in the course of just a few minutes (I was probably conducting my little experiment for 10 min?) you can see the frenzy gaining momentum then it seems to die down until only hundreds of people are talking about it per 15 second refresh.
A little bit about the graph, each tick mark on the bottom represents a refresh (15 seconds or so I think). The Y axis is the difference between refreshes.
Current Status: API Latency Issues
We are currently experiencing latency issues with the API, and we are actively investigating. We will provide an update when either the issue is resolved or we have an ETA for resolution.
DNS issues are the cause for so many outages by a myriad of prestigious web shops. It's incredible how often overlooked something so pivotal to your infrastructure goes without being properly monitored. I have seen entire businesses suffer hours of downtime because their monitoring systems would query an authoritative name server from RFC1918 IP space - but not the same from an externally visible address. An NS not responding to local queries can be just as detrimental internally as they are externally because of how web services are architected, obviously. In their defense, Facebook rarely incurs outages of this capacity, and I can bet it won't happen again and someone is getting seriously reamed. They can and will happen to the best of them, no matter how many MIT PhDs you have designing your systems.
I don't think we're uncommon. Most sites that I see plastered with Facebook embeds are publications, and I'm sure they all see results similar to ours. Facebook is RSS for the masses.
The significance of the "like" button missing from other sites is lost on me.
I've been using the Application Boundaries Enforcer feature of no-script in Firefox to block facebook content when I'm not visiting a facebook site directly.
Site .facebook.com .fbcdn.net
Accept from .facebook.com .fbcdn.net
Deny
This is motivated by privacy concerns of Facebook's ability to track which partner sites I visit.
First I just had an empty feed, then I got the feed back but posting links and comments timed out. Then it worked for a minute, and now I get the DNS error. Seems like this affects several of Facebook's applications.
I experienced a facebook outage yesterday afternoon (-03:00 UTC) although the DNS resolved the facebook I could not ping their IP addresses. As you can read in facebook's twitter timeline.
joking aside, this actually had this effect, as so many pages actually resolve some components to facebook.com, the slower response was felt on a wide variety of sites.
