Hacker News new | past | comments | ask | show | jobs | submit login
Amazon to hand over Echo audio from alleged murder after defendant consents (bbc.com)
279 points by ghosh on March 7, 2017 | hide | past | favorite | 185 comments



This should be a lesson to people who don't understand the distinction between "can't" and "won't".

If you're running things yourself and control the encryption keys required to access your data, then your service provider can't be compelled to release your data as it's not possible[1][2].

If you're delegating all of that to your service provider and they have access to the raw data, then you are putting all your trust in them to protect your data and prevent it's release. And that has to cover everything from hackers, to snooping employees, to the Feds.

[1]: Kind of ... I don't recall the Apple/FBI case going to court for a final resolution so it's possible they can compel the service provider to hack you to get the keys but at least they can't get it directly.

[2]: And obviously they can always come after you with a court order or rubber hose (or both).


Or as cperciva put it, "Playing chicken with cat.jpg"

http://www.daemonology.net/blog/2012-01-19-playing-chicken-w...


> In Tarsnap I might take this to an extreme — in addition to the aforementioned encryption, I encourage users to read the tarsnap source code rather than trusting that I got everything right

You still have to trust that what's running on your machine and on the servers is compiled directly from the source you have access to, right?


You don't need to trust the code on the server -- that's the point.

You can compile the client code yourself. In fact, until recently you had to compile the client code yourself.


ah yes, sorry.

is there a trusted compilation service that uses a distributed agreement mechanism, à la blockchain or DHT for validating some git/svn hash against a binary.

every downloadable source publishes its own binary hashes but the whole practice is somewhat moot in the event of server compromise, not to mention build reproducability.


You can compile the code yourself, then you would know the binary matches the source.

Of course, you have to trust your compiler and OS, but with a service you would have to trust their compiler and OS.


Though, of course, this still means trusting the compiler: https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomp...


True, although if someone was going to put a backdoor into a compiler, they'd have to be idiots to attack tarsnap; much better to have it trojan an OS kernel or something else which is trusted and far more widely used.


Do you trust your compiler?


Hey you guys provide binaries now? News to me!

Was never an issue till now though.


We're shipping binaries for Debian and derived platforms. Other platforms will follow.


If you build it yourself with the same compiler and options, and get the same bit-exact binary, then it's safe to assume the binary was built from those sources, since infiltrating the compiler team is not an easy job.


>If you're running things yourself and control the encryption keys required to access your data, then your service provider can't be compelled to release your data as it's not possible

The court can just hold you in contempt until you do[1]. They also probably won't buy "I forgot".

[1]: https://arstechnica.com/tech-policy/2016/04/child-porn-suspe...


If you delete the data, prior to getting the notice to produce the data and it wasn't deleted in anticipation of such a notice, sure the court could hold you in contempt, but legally you've done nothing wrong.

You cannot easily make a third random third-party delete your data.


You might put yourself in a worse position if the court compels you to produce the data, but now you can't. You better be very certain that you are able to prove that you deleted the data prior to any notice, or you might find yourself in for a long jail stay.

This is what a lot of people overlook - when the court compels you to produce evidence, or ill gotten gains, saying ,"I can't" - isn't a legal defense.


> This is what a lot of people overlook - when the court compels you to produce evidence, or ill gotten gains, saying ,"I can't" - isn't a legal defense.

We're getting closer and closer to testing that. IANAL but I'd imagine that some combination of the 4th and 5th amendments should cover that situation. The onus would be upon the prosecutor to prove that you destroyed the evidence after the fact.

As a general rule, one does not have to prove they didn't commit a crime, the prosecutor has to prove that they did. Where this gets murky is if a judge orders you to present the non-existent data as contempt of court may apply.


There's lots of case history where people are faced with a court order compelling them to return ill-gotten gains, and when they claim "I can't do it, I don't have the money." - they are found guilty of contempt of court if the court believes otherwise. Burden of evidence is not on the court to prove you do have the money, just as in this case, the burden wouldn't be on them to prove you can retrieve the data....


It depends. If you delete all data after a year pursuant to a well established practice, the court should not be able to hold you in contempt. But if your explanation smacks of destruction of evidence, then you've got a problem. Remember, it's not about what you did, in fact (the judge and the prosecutor can't know that). It's about what the circumstantial suggests you did.


Just to be clear, my assumption was the jurisdiction was US federal law, the entity deleting the data was an individual, the data was their data, and the systems deleting the data were owned by them; this based on the context as I understood it provided by the comment I replied to of this thread.

As such, would you please explain why you provide a timeframe for keeping the data prior to deleting it?


I think he's saying "If you have a policy of deleting your data older than X, you are OK." You have to be able to prove that well before being asked for the data, you had a policy that resulted in it being deleted.

Where I work, we delete emails older than 6 months as a company policy. I have to assume this is done for exactly this reason. Don't ask why we might want to do that, I'm not in charge. :-)


Right. It's about insulating yourself from the accusation that you deleted that data to cover up wrongdoing, rather than for other reasons.


There are quite a few situations where you have an obligation to not delete data so "legally you've done nothing wrong" is probably overly broad.


Right, though my assumption is the relate statements are in the context of the comment I replied to, that being an individual using only systems owned by them. As such, beyond my prior comment in this thread, I'm not aware of any law related to an individual that would bar them from deleting data at will assuming to legal request to do so was expected or in place.

Opening up to any situation beyond that would be dependent on the context of that situation.

If I am missing something, please explain explicitly what you are referencing.


Under which circumstances is an individual required to retain data that they created?


Under which circumstances is an individual required to retain data at all?


Well it would still be a whole lot better than what happens now. Microsoft, for instance, sued the DoJ recently because almost half of its data requests came with a gag order.

So you wouldn't even know when they got your data, and then perhaps used it against you - in secret (like putting you on a no-fly list, etc). And then they won't even tell you why you're on the no-fly list or why you're getting an audit.

The whole system is broken from top to bottom, and I think that has a lot to do with the fact that politicians who are supposed to represent the people, don't care about what the people want anymore or to actually represent them. They care about what various rich people or powers want them to do. It's not just me saying that:

https://www.youtube.com/watch?v=5tu32CCA_Ig


Leave the thing on a timed kill switch requiring secure input to reset the timer. It acts like a warrant canary, because the default is that without acting to intercede, your data is physically destroyed. That said... unless you have something really critical you must hide, potentially at the expense of whatever an angry/frustrated state might do to you... don't do it.

If you have an incredibly valuable idea, if you're protecting state secrets, if you're a journalist with real integrity, or a criminal who stands to lose more through exposure... then it makes sense. Otherwise, just for kicks, I don't see the point.


They can only reveal voice data IF the victim said "Alexa" when the crime was taking place. If the victim didn't say "Alexa" that day, then there's nothing to reveal. Your example about encryption doesn't make sense in the context of the Echo.


I'm not certain there exists nothing to reveal. Is there a technical limitation of the system that prevents or avoids recording of "non-'Alexa'" audio? Or do I just trust Amazon when they say their alway-on device only records when triggered with the magic word?

As I think about it, I'd ask the same regarding the voice-controllable phone in my pocket.

FWIW, I don't trust the devices, because their operations are out of any consumer's ability to control them, but maybe I'm missing something about their technical limitations that would work in my favor.


From looking at the internals, there's technically nothing that prevents uploading all audio constantly to AWS.

Practically, it would be unwieldy to have 10,000,000 devices sending audio to 10,000,000 audio decoding and processes 24 hours a day nonstop.

You'd also see a huge bandwidth hit at your ISP, which would certainly kill future adoptions if the word got out.


So record the audio 24/7 and compress and send once an hour? Once a day? Or eliminate empty space first. The files wouldn't be that big


Totally possible. As an example, Codec2 used in ham-radio applications can send pretty decent voice audio at a few kilobit/s or less, and it keeps sounding perfectly understandable even at 1.2 kbit/s like this short sample: http://www.rowetel.com/downloads/codec2/hts1a_1200.wav

More information here: http://www.rowetel.com/?page_id=452


To spell it out, that's < 12.7 megabytes every 24 hours. If there are 10 million Echo devices, that's 127 million megabytes a day, or 127 terabytes a day. That's actually not that hard to handle for the company that runs AWS, so while extremely unlikely, it's not impossible. Just very, very costly.


You don't need to record 24/7. A well planned spying operation would involve multiple devices and connections, location discovery, proximity with other devices etc. Phones, tablets, Home and laptop PCs, Car PCs, Smart TVs, and pretty much every connected device, can be hijacked into becoming a bug or cooperate with any of them if in proximity. The victim cellphone could establish a secure connection via WiFi or Bluetooth with the Echo o any similar assistant, grab the audio data to transmit, alert the user some important upgrade is needed on the phone then start transmitting the data and fake some random download just to make the downlink act as it's receiving something. That way those 12 megabytes of data would remain totally unnoticed.

This is of course the product of tinfoilhattery at its finest level, until someone does it for real.


Sure, I was just providing an upper bound for requirements if they decided to store all audio all the time from every device. Of companies that have the capability to do so under their own resources, Amazon in on the short list. Amazon could possibly pull it off and hide it in the rounding of numbers for their normal business.

Companies that interact (peer) with them would likely see something though, but possibly not as easily as it seems. The average home internet connection probably downloads far more than 12.6 MB of content from AWS hosted services every day. The only question is whether the upload amount would trigger any alarms. I think in most cases not, as it would probably just go a very small amount towards evening those peering connections out, which are likely very heavy in the other direction.


Easy answer. They may not have to send the audio. They could transcribe it locally at the client, encrypt, and send text to store on the server. Consider that almost a decade ago, programs like dragon naturally speaking could be run on a relatively inexpensive laptop. It's entirely possible that a dedicated device like the echo could do this today. EDIT: Original reply sounded too definitive


It's not transcribed locally at the client except for the wake word.


Totally not possible. 1.2kbps * 10M devices is 12Gbps, or greater than the bandwidth of a STM-64 link. Not practical to either receive or store, even for Amazon, and certainly bandwidth consumption on that scale would be extremely noticeable.


I'm not sure why you would assume one of the largest computational and datacenter service providers in the world with many datacenters in many regions would require all input to be over a single connection to a single location, and even if it was, why it wouldn't come across the many, many peering agreements they have.

There are many reasons why it doesn't make sense for them to do this, but this isn't one of them.

Edit: To clarify, and put this in perspective, 12 Gbps is 1.5 GB per second, which is less than 127 terabytes a day. Amazon, through AWS in multiple regions, is entirely capable of adding 127 terabytes of storage a day, and already transfers MUCH more than 12 Gbps. This is not impossible, just very improbable.


Not to mention that people aren't actually speaking 24/7. In any given space there are probably people talking for less than 10% of the day.


Compression over an hour or a day won't do anything meaningful vs compression over, say, a minute. There's just not that much additional redundancy to eliminate. Uploading in a big burst doesn't save on overall bandwidth, either.


I see a lot of people in this thread saying that's not possible, so here's the math:

As given by squarefoot, you can record human voice at 1200 bit/sec = 150 bytes/sec. A day is 86400 seconds, assume people are talking (generously) 10% of the day, so 8640 seconds * 150 bytes = 1.3 megabytes per day uploaded to Amazon.

Does anyone doubt that _the company that runs AWS_ is incapable of dealing with barely a megabyte per device per day?


You do realize this is same company running AWS. One of if not the largest network of cloud services. They can probably do that with idle capacity of one region.


Doesn't seem any more unwieldy than Skype or Spotify?


I don't have a link, but the echo doesn't transmit network data until the watchword is detected. It probably has a small local buffer, but not large enough to be useful here. So I expect that an unmodified echo would not provide historical evidence, but there's probably a way for one to be configured to stream audio continuously.


Unfortunately the only way to check if the above is true would be by examining the device firmware.

Also, the definition of "unmodified" and "configured" is very broad in this context. There's no need for a screwdriver or messing with a JTAG port when a well hidden magic packet coming from the Internet can trigger a watchword=off function.


Right, but this is a targeted action that can only be applied to a limited number of people without causing problems for Amazon. It's not useful when something unexpected happened and there was an echo present.


If it's true that all audio is being sent to Amazon, I will throw my Echo out today. That's a serious invasion of privacy and breach of trust.


The entire point of the Echo and similar devices is that it's always listening.


Listening locally for a trigger word != listening, recording and transmitting. The point of the echo is to have an on-demand voice channel to an artificially intelligent assistant.


The device is always listening (unless you mute the mic), but that doesn't mean it's always sending a stream to Amazon.

The device has to continuously process audio to listen for the wake word (upon which it does start sending everything to amazon) and a few seconds of the buffer before that so you can just bark out commands rather than waiting for it to acknowledge you.


The device does claim to only send voice recordings to Amazon during a query (when you say the wake word).

You can ask Alexa, "Are you always listening?" to get details about it. It's possible that the device doesn't have anything interesting anyway and they were just resisting the disclosure for PR reasons.


> You can ask Alexa, "Are you always listening?" to get details about it.

True, I can ask, but I don't know if I'd always believe the answer.


I happen to have met the hardware engineers who developed the chipset and microphone array PCB inside the Echo and they told me there is no recording unless the trigger word is detected.

I saw no evidence to prove or disprove this and there's nothing to say that the device couldn't be altered after installation.

[edited: grammar]


Once upon a time this behavior would have been determined by hardware. For a device developed this decade, however, there's no way that a PCB design would determine this. Rather, this behavior is determined by the firmware loaded onto the microcontroller.

Even if one didn't know that, the fact that the trigger word is easily changed by the user should indicate that everything is in software.


False Positives.

"I'll Let Some" one else discuss how false positives can affect detection.


Makes me think in future cases when someone says in court "I do not recall", we are going to see Google Home / Echo data being called into play. As someone with 3 google home devices in the house, I feel like my feara I've been suppressing are all coming true.


Then why did you buy 3 Google home devices?


Adrenaline rush


You need hobbies.


Home automation is an amazingly cool thing to do, and i theorize that it doesn't become a true part of your home unless you remove as many of the barriers to regular use as possible. Voice is very convenient, and access on every floor of the house was therefore something I wanted, so I bought 3.

The experiment is still in progress, but I can tell you they're 90% used for playing music right now :)


I have these same fears, yet there are 4 android devices, an ipad, and an echo dot in my home.

High utility and low chance of me being in the specific situation where internal home recordings are requested.

High downside if I do end up there, but low risk of ending up there (hopefully).


Except Amazon refused until defendant said it was OK.

It's often easier to coerce single person rotting jail on contempt charges than a huge corporation.

Point being, like everything, it is not as b/w, not as simple as you imply.


Seriously, did anyone read the article? The defendant consented, Amazon didn't want to.


Apple and Amazon also have some amount of clout that helps them stand their ground to a great degree in situations like this. The Feds realize that if they shut down Amazon or Apple, the country is going to fall to pieces.

As usual, startups are the ones that will suffer if faced with a situation like this.


I much rather put my trust in them than myself. I am clueless about security.


But in this case, even if defendant had developed a system by themselves and had complete control over the data, nothing is preventing him from turning it over as well. Here they are complying with a customer's wishes and I imagine the same person would decide to open their own data if they had developed their own system.


That assumes the data is there in the first place. What if Alexa just didn't record everything everyone said. It just sounds really really dumb.

Like snapchat saving every image privately shared on a server. It's such a big scam. Sole reason I'm not buying their shares.

I'm hoping someday we'll get true offline speech to annotated text translation that we as consumers control. Like I control what data is in my machine's hdd.


That's a good lesson, but I don't see how this example teaches it. The defendant consented to releasing the information. If Echo were designed to give the user total control over the encryption keys required to access their data, they still could have consented.


This is a non-issue. The audio Amazon receives is from only-just-barely-before the wakeword up to and through the command or timeout expiration. Yes, it's possible that the defendants cat meowed right before the murder and it sounded like "Alexa". But it seems incredibly unlikely.

Prosecutors are just trying to cover every base but the likelihood that this will yield anything is very low.


> The audio Amazon receives is from only-just-barely-before the wakeword up to and through the command or timeout expiration.

I wonder, how on earth do you know that, that you are able to dismiss it with such confidence as a non-issue. That's the problem with proprietary systems (proprietary always-on microphones at that). I would be surprised if they weren't actually listening and analysing and storing data from the microphone feed at all times.


You can just watch the network traffic from the device to see when it is streaming audio.

Now this obviously doesn't rule out the device saving stuff locally, but plenty of people have verified that it does generally only send lots of data to the server when you have just asked it to wake up.


Although that does not rule out the possibility that it stores data locally and only transmit the recorded payload (since the time of last transmission) with the next legitimate payload when the wake word is spoken.

Not saying that they do though since if they were discovered to be doing that, it's way too damaging for Amazon's reputation for it to be worth going through the trouble.


It is possible it's doing local voice recognition on all audio, and just sending the text in their payloads. If the content is encrypted, there's no way for any network snooping to determine what they're sending.


This is software... saying there's "no way" is usually incorrect.

For example, you could root the device and disable the encryption.

Or, you could perform experiments in a controlled setting where you play a series of identical recordings to Alexa and measure the statistical similarity in the outgoing data for each. The encryption scheme probably provides more entropy than just the audio, so maybe statistical analysis wouldn't help, but it's a start.


I hear this all the time and it boils down to what another poster said. I can monitor all the traffic going through my network. If the device was constantly streaming audio it would be very obvious.

To which someone once replied to me: "But what if they aren't sending traffic through your network? What if they are using 4G or something like that?"

To which I replied that while I have not personally done it. You could scan using software defined radio to detect that sort of thing. And if it was doing that someone would notice. Plus if you tore down the hardware they would notice the antenna.

To which they replied "what it it uses something you can't detect with that?"

To which I walked away because I didn't feel like explaining how physics works.


How do you know they don't

1. Listen all the time

2. Use a small neural network on the device to detect when a voice is present

3. Collect this data into one zip file, then send the file when the user says "Alexa", or anything remotely close.

They could even put a size limit on the data upload to reduce the variance to prevent you from ever testing whether they do this.

Or, they could simply transcribe the audio on the device and upload the text. Any audio they are unsure of could be uploaded to the server to be handled by a beefier neural network.


Yes, exactly this. They could easily just TTS what you are saying, save the text, and send it together with the rest of the info when you say "Alexa". Thus only sending information when you say "Alexa" but managing to upload all your conversations.

I would be very surprised if they aren't doing something like this. The power of analysing which products you talk about in your home more often, what kind of stuff you consume, what affairs do you discuss at home, etc, is too good to pass up. And seriously frightening.


It is very well known that the actually speech processing happens on the cloud. To deploy a whole cloud voice recognition system if you have distributed network with TTS capability on the device would be quite a lot of redundant work to go through.

However, with that said, unless they do certificate pinning on their device the answer to that is to MITM the device and snoop on the traffic.

If they do certificate pinning the answer is:

1. Pre-record an Alexa commend

2. Play back the recording

3. Wait a minute

4. Replay the command

5. Measure the size of the packets going across the network

6. Wait a week while playing something that sounds like natural conversation - say an audio book

7. Replay the command audio file

8. Measure the amount size of data sent between the end of the second command and the end of the last

It should be slightly more than the second command was to account for things like checking for updates. But if it includes the TTS (which is essentially an audio book transcribed at this point) than it would be quite a bit larger even with text compression.


Any amount of text - when compressed - would be dwarfed by a number of things that may also be included in the data exchange, such as a software update. There's no way to conclude that a larger exchange of data means a big exchange of a week's worth of text.


Nit: TTS is Text To Speech, so you need the other way around :) I most often saw that abbreviated with SDS, although that's also not too correct since a speech dialog system also covers more than only speech recognition.


FWIW, I've seen a couple threads on Reddit where people did network sniffing -- the echo only had network traffic after the wakeword. So, in a hearsay/amateur manner, it seems to not be doing that.

As someone else mentioned, they could of course record constantly, then compress and transmit in batches, and that would probably go unnoticed.


> As someone else mentioned, they could of course record constantly, then compress and transmit in batches, and that would probably go unnoticed.

Possibly, but it can be fingerprinted by using known audio samples and intentionally producing very large amounts of data. See my other reply.


There are too many checkers players in a chess world.

They're attempting to establish the precedent today. Tomorrow(figurative, not literal), when such devices send everything to the cloud to be processed "To reduce false positives", there will be much more data to mine.

I won't have one of these devices in my house. I won't have an xbone in my house for the same reason. Today, they're not much of a threat to privacy but long term, the risk is too great for my comfort. They can't be trusted.


> They're attempting to establish the precedent today.

At least this situation didn't give them the precedent; the defendant consented, so the court never got the ability to demand it from Amazon.


You may not have a choice, if everyone's phones are doing it.

Wikipedia leak today claims hacks for Smart TVs. Perhaps in the future, basic things like light bulbs aren't even usable without an Echo-like device.

Bet on everything being recorded everywhere, at some point. Just look at what the satellite tech companies are doing now. If you are doing something off in a far remote area, you are still being surveilled, recorded in perpetuity. At this point I'm more concerned about AI actors than any government.


Note: I think you meant wikileaks not Wikipedia


So you don't have a smartphone?


Reducing the attack surface is already a good defense, even if you have a smartphone:

https://en.wikipedia.org/wiki/Defense_in_depth_%28computing%...


I don't know about you, but I have no desire to become a digital pariah and avoid useful technologies just because they may, at some nebulous point in the future, be used against me in some nebulous way.

If and when that happens, we can fight it then. I'm a believer in picking my battles, and see no reason whatsoever to pick this one.


If all of your friends jumped off of a digital bridge... You know the rest.

It WILL be used if it isn't smothered in its crib.

It's the nature of both government and technology.

Today, it's about convenience and whiz-bang "Look at what we can do" demonstrations but if allowed to continue, it will become a tool of the surveillance state. It's too powerful a tool to be ignored.

When CALEA was up for debate, those of us who feared that it would lead to widespread surveillance were ridiculed. It turns out, we were right. We're right about this too.


Of course, you have the freedom to make this choice. However

> just because they may, at some nebulous point in the future, be used against me in some nebulous way. If and when that happens, we can fight it then.

sounds very much like https://en.wikipedia.org/wiki/First_they_came_...


You mean the slippery slope fallacy, codified into a story?


...and proved to work in real life.

"If an argument uses valid reasoning, it would not be identified as the slippery slope fallacy,[2] and the term "slippery slope" may be used without an implying faulty argument."

https://en.wikipedia.org/wiki/Slippery_slope#Non-fallacious_...


Given that there's no proof that Amazon is doing anything untoward with the Echo right now, arguments along the lines of "but they might in the future!" are an instance of the fallacious version.

If for no other reason, the behavior would be blatantly obvious to the kind of people that run Wireshark for funsies, the alarm would go up, and Amazon's reputation goes into the toilet.


Would you also say it about the NSA before the Snowden leaks? Given our current knowledge about the state of surveillance, we have enough reasons to believe in the worse.

Upd: > the behavior would be blatantly obvious to the kind of people that run Wireshark for funsies

Echo probably does not send the recordings all the time, it would be too simple to detect. However, it could simply send random parts of the recordings or send the data by a trigger from Amazon/random hackers. We can only say for sure that we have no possibility to check what it really does, since it is proprietary. I prefer not to trust for-profit companies (when it's possible). I only trust the source code.


Echo probably does not send the recordings other than the ones it's expected and advertised to send. No proof of any kind exists to even begin to suggest it does something other than that.

The device itself has been rooted via a set of contacts compatible with what you'd use with a Raspberry Pi: https://www.reddit.com/r/netsec/comments/4inesj/rooting_the_...


>other than the ones it's expected and advertised to send

and the ones that it mistakenly accepted as triggers, at least.

>No proof of any kind exists to even begin to suggest it does something other than that.

No proof exists to suggest anything about Echo. It is a question of trust. This is why I mentioned trust in my previous response.

Anyway, thanks for an interesting link.


Though the article stipulates the Echo was streaming something (music?) at the time - which could infer more 'active' attention - I also think this is more of a fishing expedition. I have serious doubts that the Echo recorded anything of value. I base this perspective of having numerous interactions with Echo devices and observing how routinely poor they are at listening in even completely silent rooms.

I enjoy the devices, but also scoff at the "OMG CSI TECH ENHANCE!!" spin that can kind of find its way into discussions via less-than-knowledgeable journalists. Or ones who want to amp it up. Whatever the case, I think it's kind of interesting to hear about.

I do pity the audio tech who might have to get the audio file, put it in a DAW and turn it all the way up to listen for, uh, evidence. Ouch.


Just a few minutes ago, my Alexa triggered on a phone conversation someone was having across the room. It happens fairly regularly. While I agree it's unlikely to have caught any specific incident, do this often enough and sooner or later something will show up - the recognition is not particularly precise.


We will have to see. I was curious about this myself, but amazon does not say categorically when it streams data to the cloud.

https://www.amazon.com/gp/help/customer/display.html?nodeId=...

Note that it says "this is when Alexa streams your voice to the cloud", but it does not say categorically that this is the ONLY time Alexa streams your voice to the cloud. I found that a lot of news articles said Alexa only records your voice when you make a request, but Amazon itself offers a more limited promise.


How is echo audio data substantially different than something like a tape recorder that was live in the man's home? What about a cell phone that was on a call at the time of the alleged murder?

I definitely understand wanting to ensure reasonable privacy for users, but to me it feels an awful lot of a stretch to say that the echo is off-limits in this case.


Comparing it to a tape recorder is interesting. If we consider this device to be the same as a tape recorder for the purpose of admission of evidence: (1) In some states, by having this device and turning it on you have now indeed consented for the conversation to be recorded - which would alarm most users (2) In other states, it is illegal for the device to be functioning without consent of all who are conversing, and the recording is thus inadmissable as evidence. Quite the opposite of what I think you're implying by that comparison. We keep seeing that treating new technology under previous legal definitions is very unsteady ground. You can see how comparison to previous technology would protect privacy and hinder investigation in this case, but on the other hand it makes me think of compelling you to give up encryption keys because well, they're not "papers" like the 4th amendment refers to (an argument which HAS been used before). So given that, how am I supposed to know what my actual rights are in a case like this, when it's so freely interpreted either way depending on the specific case?


Thanks for the insight.

In regards to comparing new technologies against previous ones to understand their legal definitions, it feels like the burden is on technologists to navigate us through the ethical minefield. I'm not very aware of what place ethicists have in many tech corporations, but it feels like most of us just build straight ahead and figure out the implications afterward. Not that we should halt progress on everything until we know if it's "good", but maybe a tech ethicist could be a kind of QA role during product development.


Definitely. I think I've heard Google has a handful of ethicists on staff? I don't think it's fair for the burden to be entirely on technologists either, though, but I don't know how else to progress. Uber & friends make an interesting case in this area too: Does that business model technically violate laws in various locales? Is it fair to existing taxi companies to just throw those laws away too quickly for them to plan or without re-addressing the original issue they were there for? Could they get lawmakers to re-assess the need for those laws before they had demonstrated their value and popularity? A lot of catch-22's.


> but on the other hand it makes me think of compelling you to give up encryption keys because well, they're not "papers" like the 4th amendment refers to (an argument which HAS been used before).

That makes me think, what if I wrote the keys down on a piece of paper, and then told you I always copy it from the paper because I don't remember it. Then would I be protected by saying that my papers are protected and therefore so is the key?


"Clearly the defendant is trying to hide something now. If it pleases the crown, might we have a search warrant to seize the piece of paper?"


> How is echo audio data substantially different than something like a tape recorder that was live in the man's home?

This is an interesting question that leads to (surprising?) ruling in certain jurisdictions. Take Germany for an example: The German law on telecommunication and privacy in telecommunication explicitly forbids the manufacturing, distribution and possesion of disguised recording deviced with transmission capability. Take note that it does not outlaw concealed/disguised recording deviced entirely, but is specific on the transmission capability. This law has been put into place in reaction to the massive invasions of privacy under the Nazi regime and until the iron curtain fell in the GDR by the Stasi (the agency depicted in the movie "The Life of Others").

A few weeks ago there was that case of the German Bundesnetzagentur (German equivalent to the FCC) banning a "smart doll" for that very reason. Now consider that certain variants of Amazon Echo could be mistaken for mundane wireless speakers, especially to people who are not familiar with the concept of connected devices. However the wireless transmission capability of the Echo clearly puts them into the reach of the aforementioned law, and some people actually argue, that because of that, they are in fact illegal in Germany. Now what about actual Bluetooth speakers with a microphone built-in to support speakerphone? Technically illegal, too, because one could put them in aunty's home, connected by wire to their CD player or such, yet use the bluetooth function to eavesdrop from outside the home.

On the one hand it's great to have this law in place. But on the other hand it's clearly ripe for being updated to match current technological developments.


> How is echo audio data substantially different than something like a tape recorder that was live in the man's home? What about a cell phone that was on a call at the time of the alleged murder?

It's not about the technology - a tape recorder isn't on the whole time while it's in your house and won't get accidentally turned on by you speaking. Alexa is listening the whole time and gets triggered even for non-standard phrases essentially sending snippets of conversations to Amazon you didn't explicitly plan to.

It's very very very rare that people press REC on a tape recorder and forget about it. It's very common for Alexa to trigger on phrases that aren't meant for it.


Just to clarify: The Echo - the hardware - is processing audio constantly to spot the wake word. Alexa - the cloud software - is definitely not "listening the whole time." Yes, the wake word spotting may occasionally trip a false positive, in which case the lights go on (and you can set it to also make a sound any time it wakes, if you prefer.)


I suspect that it is as much about PR as anything else. Amazon probably has no qualms handing over the data per se. But they also realize that doing so without first putting up a 'fight' would look very bad.


But of course. Corporations are amoral entities. They do whatever it is that they calculate to yield the highest expected profit. They will do good things only insofar as it happens to coincide with what's most profitable (as is the case). If the most profitable course of action was to murder children and they could do it without hurting profits via breaking the law or getting bad PR, they would do it (as at any rate they do in places where committing atrocities is the unpunished, most profitable course of action).


Where the data is stored is a fundamental difference.

That was one of Amazon's primary arguments, that they had not received a warrant valid in a relevant jurisdiction.


Interesting, thanks for letting me know.

Jurisdiction seems like a concept that is going to change a lot in the next decade if that's the basis for argument.


There's already procedures to deal with it.

I think it isn't the same (hacking vs a provider handing over data), but the FBI has been working to assert broad jurisdiction:

http://www.reuters.com/article/us-usa-cyber-congress-idUSKBN...


If the EULA says that Amazon's listening and you agree to be okay with that, there's little you can argue against it, other than saying don't use such devices.


I own an Echo and this is part of the reason it now sits unplugged. To be honest, it doesn't provide enough value for me to consider potentially compromising the privacy of my home.


Without trying being rude, surely this was obvious before you purchased it?


Bruce Schneier wrote a great essay on this topic: https://www.schneier.com/essays/archives/2008/01/the_psychol...

In short, we "feel" security more than we reason about it and once a risk has been personified we lend a lot more weight to it.


That is a really interesting link.


Also, your phone is probably already listening to you - and maybe your cable box.


Hell, most speakers can conceivably be used to record audio.


If people are getting murdered in your home and the cops catch wind of it, your privacy will be compromised whether or not you have an Echo.


Mentioned this in another comment, but they can only reveal voice data IF the victim said "Alexa" when the crime was taking place. If the victim didn't say "Alexa" that day, then there's nothing to reveal.


Or thought it heard Alexa. There were many many times when my devices recognized Hey Siri and Okay Google during regular conversation when no such thing was said. I've disabled that capability on all my devices as well.


The main thing I'm taking away from this is that, if I ever find myself getting murdered, I'll just keep shouting "Alexa" as a last ditch attempt at getting justice from beyond the grave. Can't hurt.


I was thinking about this -- I'm too afraid to try it, but if I said, "Alexa call 911", does that functionality exist?

Is there a skill I can install that would let me say something like, "Alexa start recording audio to the cloud"?


"Alexa call 911" doesn't work (at least currently) and it looks like it's against Amazon's rules to try and create a skill that calls emergency services: https://developer.amazon.com/public/solutions/alexa/alexa-sk...


At least in our office it happens quite commonly that Alexa will trigger on phrases that aren't the word "Alexa" and are even unrelated to it.


Is there some technical evidence that this is the case? Or is there a chance that these can be configured remotely to record everything with the right permissions without the owner knowing?


This definitely has the potential to be the "killer's name written in blood" of the 21st century.


What if someone else said "Alexa"? What if someone on TV or the radio said "Alexa"?


The person consented. Amazon did not hand it over before then.


Your phone can record, video too.

Letting fear of privacy boogyman control your life is a sad waste of living.


And "lets ignore these problems because they exist elsewhere" is a sad attempt to justify one's lack of concern for a legitimate attack vector.


> Amazon has agreed to hand over data from an Amazon Echo that may have been operating as an alleged murder took place, after the defendant consented.

A much more interesting question is if Amazon would have released the data without the defendants conset.


Well Amazon actually refused at first, even when presented with warrants requesting the data, saying that they didn't see how the data would be relevant.

http://www.bbc.com/news/technology-39063113


I know that, I'm just pointing out that handing the data with persons consent is absolutely nothing extraordinary.

Many people (also in this thread) are worried and unplug Alexa because of these news. Some accuse Amazon of putting up a fight for PR only.

From my point of view, these news add nothing substantial to the Alexa customer privacy debate. It is unclear if Amazon would have (finally) handed the data over without customers consent. It is unclear how far would Amazon go fighting for customer privacy.


It's newsworthy in this case because of the initial refusal.

Not because it adds anything substantial to the privacy debate, but because it means we won't get to see if their argument would be held up by the court.


Can you call emergency services via Amazon Echo? I've read on a car forum I'm part of about someone who was working on their vehicle without jack stands and got trapped when their jack slipped. They used Siri to call their wife and jack the car back up. They probably would have died without it.

I can easily forsee a future where consumers ask companies for voice assistants that turn on automatically when they detect duress (not needing to say "Alexa ...") before one where governments actually compel product manufacturers to do this.


For all you know, it could be recording all the time, and since it's a closed device, you can't prove otherwise.


Yes and for all you know that table you bought could include a large battery, LTE receiver, and a microphone and could be recording your every sound as well.

There's a limit to how much paranoia is warranted, and in this case I'm firmly in the "it's not" column. Yes, it could be recording your every sound for months on end and uploading it to evil amazon, but not only would they be in some hot water legally in some areas, but the media would have a fucking field day with it, and for what gain?

To be able to listen in on your conversations?

And if you want to argue that it could be remotely updated to target you to always record that data, you could always be targeted by inserting a microphone into anything else of yours (good 'ole bugs).

You don't need to like them, you don't need to own them, and you don't need to be somewhere that has them, but some of us see utility in devices like this. And to me it's well worth the trade off that my recordings could be used in a court of law when requested with a warrant and I agreed to it.


The comparison between a bugged table and a device whose primary purpose is to always be listening to you is a bit of a stretch, don't you think?

Amazon provides cloud storage and processing to the web. Given their history of relatively ruthless (if strategically smart) business moves, I don't think it is unreasonable to consider that they could store this data for use down the line or do so at government request. This is all software based permissions that seem to be one invisible server side tweak away from becoming an always recording (not just listening for a wake word) device.

I got an Amazon Tap because it required a button push to record. They just enabled the option for always listening just like the echo. In theory I control it, but clearly I do not in practice.


To be completely honest, the "save this at government request" is a real threat, but there is nothing that amazon can realistically do to prevent that (short of physically not manufacturing the device to have a microphone, but even then is it that much of a jump from "forcing them to write code" to "forcing them to add a microphone"?)

But in terms of adding this kind of surveillance stuff for greed? I just don't see it paying off. Regardless of how shady or ruthless you think amazon is, they aren't going to brazenly break 1-party and 2-party listening laws. And all it takes is one guy somewhere who owns one to discover it and it's all over.

People are more than willing to give up information for very little gain, there's no reason to try and "steal" it illegally. If the argument is that amazon is a greedy company willing to do unethical things for money, where's the money in this? Where's the money in 24/7 audio recordings vs recordings of when you are speaking to the thing?


Your android alarm app or iOS app could to be listening and uploading as well. So...


Actually you can monitor the packets coming through your network and prove whether it is recording or not.

Also reverse-engineering the Alexa, and taking it apart will allow you to see whether there is enough storage space for lengthy conversations. Which are only stored for in-transit post the wake-word however.

You can delete your recordings on amazon.com and google also allows you to delete your recordings if you use google home. So this will give you insight into what recordings they do have.


> So this will give you insight into what recordings they do have.

This will give you insight into what recordings they tell you that they have.


Well the culmination of both datasets will allow you to decipher if what they tell you is what you sent.


At this point I assume they have a warrant, so I don't understand what the problem is. People should assume their devices can and will be used against them. A much more tricky conversation is when a friend of mine discovered that fragments of his a session with his psychologist recorded by his Android Wear watch.


> people should assume

What I assume is that the bill of rights will not be subverted just because I use technology. The right to not be subject to search and right to not self incrimate.


It's a nice sentiment and sounds like a good legal argument. However, your constitution doesn't do me any good and doesn't help you with foreign states. Five Eyes has been a legal work around for years so keep fighting the good fight, but in the meantime I will keep expecting the best and assuming the worst.


> What I assume is that the bill of rights will not be subverted just because I use technology

Given all of the revelations in the past few years, I'd consider that dangerously naive. Sure, the BoR should settle the argument, but it has notoriously been ignored.


Can you share more details? How did they discover it?


Not OP, but I'd guess through Google's Voice & Audio Activity page [0].

Mine happens to be littered with expletives directed at Google, from self-psychotherapy when my pocket computer false-positives my voice, or most surprisingly and infuriatingly, silence.

[0] https://myactivity.google.com/myactivity?utm_source=help&res...


I don't have Android Wear. But as he explained it he went to the Google personal data page that shows you the data they've collected on you and said there were audio recordings and transcripts with words and phrases that seemed odd but then he realized where he said them.


> after the defendant consented


But how long before lack of consent is considered incriminating?

There are also many different jurisdictions which don't have 5th amendment style protections.


In the UK, we have the right to remain silent when questioned by a police officer. But in 1994, the wording was changed to include: "You do not have to say anything. But it may harm your defence if you do not mention when questioned something which you later rely on in court."

https://en.wikipedia.org/wiki/Right_to_silence_in_England_an...


It will be a long time in the jurisdiction of this case before constitutional protection against self-incrimination is stripped.


That protection has mostly gotten stronger over the past 200+ years, so I think you're probably right.


Wouldn't a jurisdiction without 5th amendment protections be unconstitutional?


We do not all live in the USA.


Most jurisdictions have similar provisions. If not, the defendant is screwed anyway.


This case does though and considering its a 5th amendment case I'm not sure how it would be relevant internationally.


The basis premises (privacy amidst a sea of devices which record at various times and save these recordings off-site) are universal.


Depends on the constitution.


Feels a bit weird. Why does a defendant consenting change anything? Presumably if they were asleep as claimed, they're basically consenting to something that's unrelated to themselves.


Even if they were sound asleep; what Echo recorded may be incredibly damning - a muffled voice, screams, no evidence of someone sleeping nor of an intruder, yet enough to pull a Jury's emotions that a conviction is needed and the death penalty is merited.


Yes, this sounds like agreeing to a police interview without your lawyer present -- there's almost no chance that it will help you (are they going to hear "Alexa, tell this person who is not James to stop murdering me"?) and basically any audio they hear that's not someone else confessing to the crime can be used against him in court.


That was my first thought. Why should the defendant have a say in what evidence can be collected? Did the Feds ask if they could take DNA samples from the scene, for example?


In this case it is because Amazon resisted the request from the legal system for the data.

The owner of the device giving consent to Amazon to release the data avoids having to play out the legal process of obtaining the data (it's likely the request would eventually have succeeded).


In this case, the defendant may have a say only because Amazon is claiming there is standing to enforce the defendant's First Amendment rights - the so-called chilling effect.

With the defendant consenting, Amazon's standing for litigating this issue becomes more fraught.


You could allow the police to do that without a warrant, if you wanted to. Amazon doesn't need to argue whether the prosecution have a legitimate order to seize the data if the 'owner' is willing to turn it over.


Why did the defendant's lawyer allow this? I don't understand how this could possibly help the defense.


Well at least one explanation is that the defendant is actually innocent, and thus isn't worried it will be incriminating.

Another explanation is the defense is sufficiently convinced that nothing conclusive will be found anyways, and think that acting as if they have nothing to hide will make them look more innocent.


Most likely Amazo Alan already allowed the defendant to review the data and they've found nothing useful.


The power difference between the state and an individual invalidates any consent.


I thought Amazon claimed this was impossible and they didn't have this audio?


If the defendant consented, couldn't they just log into their own account and get the audio?

In the app, you can hear back all the queries you made to Alexa.


I haven't seen a compelling use case for these things yet, including visiting the homes of people who own one. But I sure have seen a lot of deal-breakers.


I know someone who is smart and reasonably concerned about privacy issues, but she loves the thing - uses it instead of a shopping list. I was quite surprised that she would compromise for such a trivial convenience, but it made me revise the likelihood that they'll be more widely adopted.


Seriously, screw google home and Amazon echo. If I wanted a wiretap, I'd go to FISA court.


Odds are you already carry one around with you every where you go.


It does not matter whether he carries, there lot of people around him who would carry. And for that matter i dont even believe regular land line phone.


So we should ignore one attack vector because another one exists? sounds like a pretty poor argument to me.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: