Not to mention smartphones seem to be explicitly designed to not provide this use case. Where is my always listening Google Now? If I have to take my phone out of my pocket and unlock it, I may as well type the question myself.
The Moto X and its successors have had this feature for a while. Your phone can be sitting on the other side of the room, completely off, and you can say a certain catch phrase to activate Google Now.
I can usually activate Google Now from a few feet away on my Moto X (2013) but it can't understand what I'm asking after that. I have to get much closer for it to work.
Makes sense I guess. The "listen for the trigger" bit is done locally on those Motos (I've got the 2014 variant) but really all it needs to do is keep an "ear" open for that one phrase so it's probably got a better margin, only helped by local processing. The bit of audio that comprises the query has to get compressed and interpreted on the server side (not to mention the fact that it consists of a much wider variety of possible content to be parsed) so that can't help. I guess a dedicated chip that's just constantly asking "did anyone say 'OK Google?" has an easier job than a remote process asking "wtf does that fuzzy bit of compressed audio mean?"
(obviously being silly and oversimplifying but same idea)
All recent Nexus phones have always-on hotword support. I just tested it from 20 feet ("OK Google, what's the weather like tomorrow") and it worked fine.
