I believe searching your car and searching your person constitute two entirely different searches. If your wallet is on the seat next to you, the police could theoretically use that as grounds for a search. If your wallet is in your pocket, and they take your wallet off you as part of a search of your vehicle, you'd probably have a better chance of successfully claiming unreasonable search and seizure.
I'm not a lawyer either though, so take that for what it is worth.
In many places you won't have a chance of doing anything. Just do a quick search in youtube and watch all the people being robbed, beaten, raped, and killed by the police. That's the power they have now.
I'm not a lawyer either though, so take that for what it is worth.