In a way, it's counterproductive to limit VSZ. It's perfectly fine to mmap() gigabytes of files to virtual addresses, but you neither expect nor want to have that counted against your "memory usage" limit.

The entire problem is, in a way, unsolveable with current OS APIs. AFAIK there is no preexisting, good, actually usable, and universal memory usage meter. Some things work for a lot of cases, but I don't think there's anything that would work universally.

Coincidentally, in my eyes the best way to handle memory pressure in applications would be proactively rather than reactively. Sometimes you can unload more things, like pages in a document that aren't being edited right now. And if you need to fail, you can fail on good boundaries (e.g. refusing entire requests in server-like things) rather than in the middle of something that you need tons of work to unwind correctly.

(Slightly related: pool memory allocators, e.g. APR's https://apr.apache.org/docs/apr/trunk/group__apr__pools.html )