Could it be that the user simply learns to press a key slightly earlier to compensate for the latency? There is key travel time you have to account for anyway.
Rythm games almost always have a calibration setting, where they ask you to press a key on a regular beat. Then can also do it to check the visual latency by doing a second test with visuals only. This allows them to calculate the audio and video latency of your system to counter it when measuring your precision in the actual game.
Oooh, that explains why when watching some super high bpm runs I always got the impression that they were like a frame off the marked area - but it was just shifted ahead of the actual music and they were in actually in sync
