After going out to see a movie for the first time in a few years this past weekend, I can't really picture many of the overweight families I saw fully reclined shoveling hamburgers into their faces wanting to get up and "interact" with a movie.
But could you picture a set in the round with all of the audience sitting in a circle in a tiered auditorium, and a set being rendered over the audience and actors performing in the middle?
Shakespeare in the round would be given a fully immersive experience, and it didn't require any person to move at all... it simply means that from the perspective of each individual the set is fully complete and hides the audience as much as possible and gives each observer a unique angle, focus point.
That an actor could interact with a modern object (foam stick) that is rendered as an epoch suitable object (a longsword) would be possible.
The possibilities are really quite something, for the scenario of "each viewer has a lightfield viewer and the scene, set and objects can be rendered in real-time for them".
Even if you could solve the tremendously difficult technical problems, that's just not an experience that mass market entertainment audiences would want to pay for on a frequent basis.
