Not if they just made the site return a 404/500 just to his IPs, which any half-decent adversary would do. The "play dead" strategy works great with these kind of vigilantes.

We're employed similar tactics against DDoSers at work. Start returning 500s or just tarpit their requests, they think the site is down and they go home.