The flip side of the coin is that as soon as you're pushing more data than the link can hold, if you encounter incompressible data (say I just generated some recursive high-frequency Perlin noise) - you would start getting stutter and lag.
Yeah, it couldn't be perfectly general. You could still keep the decoder simple and just put a little more intelligence on the GPU side to handle quantizing on the time/space/color dimensions appropriately when you try to send a pathological image. Getting the few multiples of compression you'd need to run 4k@48bit 240hz on current interfaces would probably be safe, trying to put HD over a serial cable will probably go less well.
