If you don't trust Apple/Google enough to have one of their standalone assistants in your home, how can you trust them enough to believe disabling the workalike assistant on your phone actually truly disables it?
It has a special circuit that can literally only recognize the wake word. You have to trust that the circuit does that. Same as the "disable Siri" button on your phone.
Phone battery life creates a constraint on the phone's ability to perform constant surveillance. An assistant that plugs into the wall lacks this constraint.
Apple isn’t making the smart thermostats, google is. I had a lecture at school where nests business model was in part to monetize home owners electrical and heating use data (eg, to optimize electrical grids).