It honestly depends on what you are rendering. If your application is just pong, it is feasible to implement a rendering pipeline that is virtually instantaneous. GPUs have overhead for even the most trivial application. With flip-flops or other analog circuitry, you are talking about frame latency in terms that are similar to the propagation speed of an electron through arbitrary copper wires or semiconductors. If you insist on digital output, some buffering will be required, but in the analog output case you are probably looking at a handful of microseconds of latency between input and output.
The available fpga hardware is just not able to compete with the sheer hardware power (ghz,ram,gates) of a todays gpu - but i would agree on smaller problems
FPGAs can achieve very compelling performance when programmed properly, but usually not for compute bound workloads. Most interesting problems are memory bound these days, and that's where FPGAs shine, since you can design your hardware to be as wide as your problem. Or alternatively if you have many parallel sub-problems, you can make your hardware as narrow as the sub-problems and replicate it many times.
You can usually move data on and off chip very quickly also, since high end modern FPGAs have many hundreds of pins. Generally these get connected in to hard-logic like fiber networking or PCIe.
An FPGA soft core is never going to beat an ASIC if both were designed well. But they can beat general purpose ASICs (e.g. GPUs) for certain classes of problems, mostly those where you can exploit the massive memory bandwidth of the FPGA.
I think that for rendering computer graphics, you really just want a big fat pile of FPUs. FPGAs will usually have a number of hard logic DSP blocks on-die, but nowhere near as many as a GPU. If rendering 3D graphics is the problem you want to solve, you probably really want a GPU.
