It works as you described: Texturing the mesh "live" as you move through it. It does use Three.js as a base, but needs custom shaders to make it 60 FPS.
Matterport works special magic on the transition between 360 images in their SDK. As far as I understand it, they render 360 cube camera onto the material of the mesh of the environment as you move so it looks like you're moving in the real mesh but only seeing the 360 image. I tried to approximate this myself in Three.js but didn't get anywhere near the quality or performance of their work. Homage.
