There are a few factors involved. If this is your personal checking account you're almost (lol) 100% safe.
If this is a merchant account which it needs to be if you're pulling from other ppl's accounts then yes. Hence the very strict consent rules. At the end of the day there's no guarantee of course but if it comes to litigation a recorded message with a verbal approval might tip the scale in your favor. Most merchants don't bother of course and just write this off as a cost of doing business and recoup the funds by factoring fraud costs into their prices.