>> When we are legally obligated to comply with demands, we pull the specified content from our servers where it sits in an unencrypted state...
Why would the public's private data sit in unencrypted state on Microsoft's servers? What would be the point of encryption if corporation servers can see what you think you are securing via assumed privacy?
I'm not being snarky but what gave you the impression it sat there encrypted?
(caveat: I'm not a security expert) Encryption is used for data 'in flight' as well as 'at rest'. As far as I know, very few companies/services go out of their way to encrypt your stuff at rest (it's within their systems/firewall/etc at that point). Even if they did, they hold the encryption keys so can 'see' it anyway.
If you want a situation where a company cannot see your data, you have to hold the encryption keys yourself (nb: knowing the password != holding the encryption key. My rule of thumb for this is if I can do a password reset on a service, then that service can see everything I put into it.)
Yes, although there's a cutting-edge form of encryption called homomorphic encryption that would allow full-text search on encrypted data if an implementation is ever perfected. IBM is working on it, among many others:
Erm, no. It's actually quite easy, if you're prepared to do a little more work.
You just create a search index before encrypting the data. Then you encrypt the index. Each time you need to search, you decrypt the index, get a reference to one or more results, fetch those, and decrypt them.
That's trivialising what can become a pretty complex scenario, but it illustrates the point.
Actually, I did an email archiver as a side project, I still have the Lucene index of my emails (including searchable attachments, of course). It's 233MB. How long do you think it would take to decrypt it with AES256 everytime I search for something? How long it takes when you search in Gmail? Can you point me to any implementation that does it like that? Or any implementation that does it at all?
I wasn't saying it's impossible, just that it's pretty hard.
I don't know of anything open source. Initialising the AES crypto provider takes a lot longer than decrypting ~250 Mb data, so keep it initialized for the duration of the session.
That said, I did it in a native client app, where state is easier to maintain.
How many consumer-focused Internet companies encrypt user data in storage by default -- in a way that the company itself cannot decrypt given a lawful court order? (I'm not talking here about transit HTTPS/TLS encryption.) I don't know of a whole lot.
>> When we are legally obligated to comply with demands, we pull the specified content from our servers where it sits in an unencrypted state...
Why would the public's private data sit in unencrypted state on Microsoft's servers? What would be the point of encryption if corporation servers can see what you think you are securing via assumed privacy?