That seems like a cq based encode. Try av1an in its target quality mode - I've gotten some movies to a tenth the Blu-ray size with 98 vmafs and with only smudged detail at pixel level scrutiny.

The trick is if your encoder can adapt the quantizer based off a metric to compare the encode quality you can save a ton more space without visual degradation.