The one thing that might be of some small comfort to others is the fact that at least the NSA didn't know about this bug.

If they had, there wouldn't be any talk of HTTPS being a barrier in their leaked presentations.

That said, all of that encrypted traffic they've got stored up can now, thanks to this bug, be decrypted.

I would think that when they discover a superpower they compartmentalize that information.

You could make the inference that, if they did have it, they were not making broad use of it.

They did compartmentalize. They still do.

That's why having an insider with sysadmin access and prestige for social engineering purposes is so dangerous.

Yeah. We should still take Rumsfeld's advice about what we don't know.

Sure, the advice is still as sound as it ever was.

So was the advice about going to war with the Army you had, not the one you'd wished you had.

The bug was introduced in version 1.0.1, released 14 Mar 2012. Aren't most of Snowden's documents older than that? And we certainly don't know what the NSA has been doing since Snowden collected his files late 2012 / beginning 2013.

It could have been discovered them between the presentation and now.

Everyone should be adopting perfect forward secrecy in their TLS connections. No excuses at this point.

In case the bug was really actively exploited on Heroku, updating private keys and changing Heroku password can be not enough. SSL endpoints forward all HTTPS traffic, so according to my understanding cookies, application specific passwords, basic auth credentials could have been potentially compromised.

Another potential vector are add-ons that use encrypted connections (for example Heroku Postgres). These also could have been targeted, in which case add-ons credentials would need to be regenerated.

That is my understanding as well. Any memory in the same process as OpenSSL could be compromised. In the case of Heroku that includes any traffic to/from your app, but not dynos' memory, so internal application credentials should be ok, unless those services were also vulnerable. Hopefully Heroku is coordinating with add-ons to determine that and automatically regenerate affected credentials.

Loading an https://APPNAME.herokuapp.com page, I'm seeing a certificate (sn:"0E:3E:94:7F:C0:64:D7:4A:52:B1:38:D7:71:90:88:1F") with an "Issued Date" of "1/20/14"... which doesn't sound like it's been regenerated in the last 24 hours.

Am I interpreting the certificate info wrong? [edit per official answer below: YES] (Are fresh certificates sometimes given much older start times? [edit: YES])

This blogpost doesn't clearly address *.herokuapp.com HTTPS service, as opposed to the custom-domains "ssl:endpoint" service.

Hi, Heroku engineer here. We rekeyed our certificates, including the one for *.herokuapp.com, meaning we resent our original certificate requests (CSR) to our CA but signed with new private keys. This is why the dates didn't change.

Our CA will eventually revoke the previous incarnations of our certificates, signed with the old private keys, making them invalid.

Am I missing something, or doesn't the browser have to explicitly check for key revocation? I know the checkbox was off in my version of Chrome. Maybe I set it, but I'm not sure.

This [1] Seems to suggest that Firefox, for instance, only checks for EV certs?

[1] http://news.netcraft.com/archives/2013/05/13/how-certificate...

So is there a simple way to check if an arbitrary site updated their certs?

Thanks! It might be reassuring to others who see the same mixed-signals if the official blogpost made specific mention of APPNAME.herokuapp.com HTTPS, and that the certificates may look older but really are fresh.

Thanks for the feedback, we have updated the blog post to include some information regarding this.

This is completely the wrong approach. Your private key might have been compromised and you're generating another certificate for the same compromised private key? What is that supposed to do?

As discussed on twitter (https://twitter.com/grittygrease/status/453606054698692608), I should have said:

We generated new CSRs, with new private keys, but with the same dates and details as the originals. This let us get fresh certs without going through a full renewal.

Thanks for the prod.

Thanks for the update. Happy revocation day!

I think you misread... the poster said they used a new private key with the same CSR.

The CSR contains the public key for a unique private key. How do you have a new private key with an old CSR?

Several certificate vendors reissue certificates with the same "not valid before"/"issued" date as the original one.

Yep, I was just hassling our billing SaaS provider about this, but then I looked into their SSL vendor heartbleed post, and their customers mention this in the comments:

http://blog.digicert.com/2014/04/heartbleed-openssl-fix/

Excerpt: "Also, Busy IT Guy is right; it's a little disheartening to be listed as Unsafe after having done all the right things, just because the issue date didn’t get updated."

I'm pretty sure if you're using herokuapp.com then you're not using your own cert and Heroku will update it for you. If you're using ssl:endpoint then you will have had to submit your own cert, and that will need replacing.

Indeed, that's my expectation with herokuapp.com.

However, this blogpost says, "As of Tuesday, April 8 at 15:55 UTC, all Heroku certificates and infrastructure have been updated and are no longer vulnerable"... while the certificate still superficially looks like it's almost 3 months old. Something isn't as it seems, I hope it's just the certificate age.

"This vulnerability can be remotely exploited to leak encryption secrets from Heroku applications, allowing an attacker to retrieve the private key used for SSL encryption and decode data obtained by intercepting traffic"

Does anyone know if this is "actually" true?. By looking at the bug it seems you can dump up to 64 bytes at a time from the stack. Given that the attacker doesn't control were from the stack at all and looking at the code the top of the stack is probably holding some random structure, is it really possible for the attacker to retrieve the private key?

I just want to be sure before spending a bunch of money to replace re-issue all certs.

64k, not 64.

OK fair, 64K. But is it actually possible to control were you get the 64K from or is it just off the top of the stack?

And is it actually possible that the top of the stack has your private key?

It has been repedately demonstrated that it is possible to get private key from the server using this vulnerability, as well as user login details. You don't have only one shot - you can query the server multiple times (and every time you will receive slightly different data) until you have everything you want. So yeah, the issue was/is very serious.

See this post for a trivial example of session theft from JIRA [1]. You may not get the memory chunk you want the first time, but you can repeat it until you do.

[1] https://www.mattslifebytes.com/?p=533

It's from somewhere on the heap not the stack. You can influence where on the heap the data comes from by adjusting the size of your request, amongst other tricks.