That's an interesting idea: certainly worth experimenting with. As you say, with GPU passthrough it ought to be possible to get a near-native experience on the virtual desktop.