Have you tried out HoloLens, Oculus etc? Technology is simply not there yet. Field of view is very limiting. Resolution is very low. Hardware is highly optimized for graphics, not high performance computation. None of the devices I have tried could be worn for more than 30-45 mins before you start having headache (because your brain is seeing depth but your eyes are on different focus plane). VR/AR tech probably needs 5 more years of advancements (if not more) before your scenario becomes reality - but even then there are significant physics barriers to cross.