That was a bad and unnecessary feature but the privacy benefits of running a model on device rather than in the cloud are undeniable.

The fact that they shipped it shows they don't know what they were doing, private or not.

That’s a little unfair imo. Statistical models make mistakes and have failure modes which are difficult to predict.

When the bug popped up, turning the feature off was easier than retraining and redeploying.