I guess our firewall configuration must be different. What are you using to run VMs, if that's your use case? I'm using proxmox, but I'm curious to hear what your setup is like.
Bare metal at Strato. I got fail2ban in place, but still - for services that you need to have reachable on the Internet on standard ports (web, asterisk) they normally should be plagued by automated scanners the second you open the ports.
Too bad that it never got implemented in DNS to specify the ports there (e.g. example.com would have an A record for the IP address and another record for the port)...