I've seen "problem baboons". The issue with baboons is that they are clever and dexterous enough to do things that other wildlife does not. They know to pull on door handles. So they can get into a kitchen via an open window, open cupboard doors until they find the one with the cold food inside, leave the door open, haul the food out, eat some, shit on the floor, climb up the drainpipe onto the roof with some of their haul etc.

They're clever enough to scatter when someone raises a rifle and points it at them; but they'll do the same if you use a wooden walking stick.

And of course they're highly social, seldom encountered alone.