'these groups of people' does imply there are (other groups of) people immune to it, whichever group they believe themselves to be in.

(That's an everyday speech 'imply', not a mathematical 'imply'; where of course the existence of sets containing x says nothing about the existence of sets not containing x.)