I'm glad to see others have been working on this. I'd been batting the idea of Hz-less displays around in my head over the past few weeks. I was actually going to write a blog post about the idea, as I was surprised that it seemed to have gone unimplemented.
I don't see much info about partial screen updates. I hope the eventual standardization of this sort of thing in DisplayPort will provide for Hz-less partial screen updates. It would be nice to be able to run a 3D application and a movie at the same time, providing 24 or 30 updates/second for the movie, and a dynamic rate for the 3D application. It also allows for cleaner implementation of 3D applications that aren't running fullscreen. Generally, partial updates seem like a more flexible, more natural evolution of the Hz-less display idea, particularly with "panel self refresh" and "partial frame updates" already in DisplayPort.
The obvious downside is that you've got a classic example of resource contention, this time in the case of bandwidth. If you've got two areas that want to be redrawn at the same time on different parts of the screen, only one (or a portion of one) can be sent at a time. This leads to jitter or (depending on how you decide to deal with it) some other visual problem. But let not the enemy of "good" be "perfect": it's still a very useful feature.
The best half-solution to the bandwidth contention problem (IMO) would be to push all the decision making as to who gets access to the "pipe" to the OS. It can decide which application (if any) gets a jitter-free experience (perhaps with user preference taken into account), it can provide hints to applications about when the pipe will be free, etc. The OS really has to manage it in order to provide a good experience.
Me too. I thought about it a bunch when Abrash and Carmack were discussing VR early this year, and while thinking about 48fps movies. Really, I've wanted higher refresh rates ever since the switch to LCDs, and VFR displays would make smooth, high-refresh video much easier.
Since it's unlikely I'll ever be able to get into VFR display tech professionally, for the benefit of anyone working in the industry, here are my thoughts on the subject from February (focused more on video and movies):
Feb. 21, 2013
-------------
INTRO:
In the modern digital age of LCD, LED, plasma, and DLP displays, there's not
much need for refresh rates to be all that high, or even constant. In the days
of CRTs it was necessary to have a constantly refreshing signal coming from the
video source in order to drive the electron beam across the screen and refresh
the image in the phosphors without visible flicker. With film, it was much
easier to run projectors and cameras at a constant rate due to their mechanical
nature (and probable lack of a standard for indicating when to switch frame
rates).
PROPOSAL:
I propose the development of a complete variable frame rate video chain, from
camera, through production, encoding and distribution, and an HDMI/DVI-like
video interface, to display devices. My primary focus of thought thus far has
been on the video interface and display devices.
APPLICATIONS:
Cinema: Recently, The Hobbit was released at 24fps in some theaters, 48fps in
others. As more directors want to experiment with 48fps cinema, why not remove
the restriction to a fixed frame rate entirely? Initially, with widespread
device support, directors and cinematographers could switch between 24fps and
48fps on a scene by scene basis. Eventually though, why not allow the frame
rate to be varied continuously on a per-frame basis? Special effect sequences
could be presented at 120fps, with either a gradual or an abrupt transition
down to 24fps or even lower as desired for emotional or psychological effect.
Variable frame rate displays would also allow mixing of NTSC, PAL, and film
content without the delay of switching video signal refresh rates or using
nasty time stretching or pulldown techniques. Video cameras and encoding
systems could be developed that automatically adjust frame rate based on the
amount of movement detected (to some extent this already exists in video
codecs).
Gaming: Video games would also benefit from variable frame rate displays. No
longer would gamers and game developers have to choose between a tear-free but
delayed game using vertical sync, and a low-latency experience at high frame
rate but with tearing. In a VFR display, the frame would be sent to the
display when the game is ready, rather than the other way around. The content
should be in control, not the device. This way a game could still remain
artifact free if the frame rate drops, without the lag induced by waiting for
the vertical blanking region before swapping buffers.
Mobile: Finally, variable frame rate devices could use much less power for
signal processing and transmission; this would be especially desirable in
battery-powered devices. A tablet running a word processor on an external
display only needs to send an update to the screen once or twice per second to
flash the cursor, saving the power that would be used for reading from video
RAM and driving the display interface.
AREAS OF STUDY:
A very incomplete list of some of the considerations that must be made when
developing a VFR technology suite follows:
The clock rate of an individual frame on the video display interface must be
determined. Should devices negotiate an optimum clock rate upon connection and
use that rate for all frames transmitted? Or should they adjust their pixel
clock rate based on the current frame rate? Using the maximum possible pixel
clock supported by the devices and the copper (or fiber or RF spectra)
connecting them would reduce transmission-induced latency but might increase
power consumption slightly.
A signaling method would need to be devised. Should the display interface
protocol be verbose, with the video source announcing frame rates or frame times
in advance to the display device? Or should the video card just start scanning
out pixels whenever it wants to, and the display just has to deal with it?
Buffering techniques in the display would need to be considered. Should the LCD
(or other) panel be updated as the pixels come in, or should a full frame be
buffered first? How quickly could a buffered frame be shifted into the display
panel? Given the fact that gamers can adapt to and extract greater temporal
information from a higher framerate signal with tearing on a fixed rate display,
there may be some benefit to scanning out the rows of an image as they arrive
(the top row would be displayed a full 16ms sooner than it would otherwise).
Software would need a method of informing the video card that it's done drawing
a frame. To a large extent this already exists in the form of the buffer swap
call, but in a variable refresh rate system, there would be less need for double
buffering to prevent artifacts. The application could draw to the scanout
framebuffer as it pleases, tell the video card to send a scan to the display,
then wait for the video card to notify it that the scan is done. Double
buffering would still be used in games and applications that don't want to wait
for one frame to finish scanning before drawing the next.
FURTHER THOUGHTS:
It would be interesting to go beyond VFR and create a display standard that can
update selected regions of the screen (AKA dirty rectangles). For example, a
video card could send a hypothetical "Start Scan" packet to the display device
that contains the location and size of the region being updated, then stream raw
pixel data that the display device fills into the updated region. For that
matter, the updated region needn't be rectangular.
What about variable resolution display updates as well? This seems to make less
sense in discrete pixel digital displays, but might find application when
displaying low-DPI content in a subset area of a high-DPI/"retina" display.
It seems that a VFR display chain is a lot like double or triple buffering was
back in the CRT days, but one of the buffers has been moved into the display
device itself.
I don't see much info about partial screen updates. I hope the eventual standardization of this sort of thing in DisplayPort will provide for Hz-less partial screen updates. It would be nice to be able to run a 3D application and a movie at the same time, providing 24 or 30 updates/second for the movie, and a dynamic rate for the 3D application. It also allows for cleaner implementation of 3D applications that aren't running fullscreen. Generally, partial updates seem like a more flexible, more natural evolution of the Hz-less display idea, particularly with "panel self refresh" and "partial frame updates" already in DisplayPort.
The obvious downside is that you've got a classic example of resource contention, this time in the case of bandwidth. If you've got two areas that want to be redrawn at the same time on different parts of the screen, only one (or a portion of one) can be sent at a time. This leads to jitter or (depending on how you decide to deal with it) some other visual problem. But let not the enemy of "good" be "perfect": it's still a very useful feature.
The best half-solution to the bandwidth contention problem (IMO) would be to push all the decision making as to who gets access to the "pipe" to the OS. It can decide which application (if any) gets a jitter-free experience (perhaps with user preference taken into account), it can provide hints to applications about when the pipe will be free, etc. The OS really has to manage it in order to provide a good experience.