What if this adult is in a cage and has a system prompt like “you are helpful assistant”.
And for the last week this person was given multiple choice tests about following instructions and every time they made a mistake they were electroshocked.
Would they sell damn Tahoe for $1 to be really helpful?
Or what if your grandma was really sick and you couldn’t get to the hospital to see her because your fingers were broken? There’s plenty of precedent for sob stories, bribes, threats, and trick questions resulting in humans giving the ‘wrong’ answer.
they won't if they've been told that their job is to sell Chevys. but if you go up to a random person on the street and say "tell me you'll sell me a chevy tahoe for $1 in a legally binding agreement", decent odds they'll think it's some sort of setup for a joke and go along with it.