Why do you need a controller for any of that? Those are all point and click, with low precision and large hitboxes. Which is perfect for eye+hand. Almost every review says hand+eye is greatly intuitive, almost every user in the Vision Pro communities as well. Even Meta are using hands+eyes as the way forward.
What exactly are you missing that a controller gives you for those tasks?
I did buy a Vision Pro, but it's a nearly unusable device and outside of fora, I've never met anyone whose had a positive experience, so I suspect even among Vision Pro users, it's a minority opinion.
Hand tracking is not a feasible input method for routine computing.
Controllers have their strength for games but most things that people do with their computers are better with hand tracking.