The N64's "GPU" is the Reality Display Processor (RDP) chip which is specialized hardware built by Silicon Graphics specifically for the N64 console. The RDP runs a custom program that is part of each game that tells it how to take high-level commands from the game code and render them. For this accurate N64 emulation, it is being emulated at a lower level than before.
Previously, N64 emulators just used the commands sent to the RDP to tell the host system GPU what to do with something higher-level like OpenGL or DirectX (of course, this meant a lot of game-specific "hacks" in the emulator), rather than emulating the RDP itself by sending lower-level commands directly from the RDP to the host GPU with something like Vulkan. This is so-called High-Level Emulation (HLE), and it's a massive shortcut to emulating the whole RDP -- which meant that N64 could even be "emulated" on a PC from 1999.
Lower-level emulation of the RDP itself has been recently made possible, and now it can also now be "up-scaled" to arbitrary resolutions -- instead of just sticking to high level emulation and telling OpenGL or DirectX to render to a larger resolution -- or even worse, scaling the rasterized rendered output frames by treating them as images.
In practice, Mario 64 was just telling the RDP to render triangles anyways so it mapped nicely to OpenGL for the HLE case, but for more "accurate" emulation (this is like getting from 95% to 99%), the RDP itself needs to be emulated as well for things like the Perfect Dark drone camera mentioned elsewhere.
To be pedantic the RDP was the rasterizer (fixed function) which was fed by the RSP processor. The RSP ran the display-list commands from the cpu/memory (very similar to a Vulkan command list) and did vertex transforms/clipping before feeding its output to the RDP.
Most games used one of a few Nintendo provided RSP programs although later in the machine's lifetime they opened up the RSP compiler and tools to developers.
Previously, N64 emulators just used the commands sent to the RDP to tell the host system GPU what to do with something higher-level like OpenGL or DirectX (of course, this meant a lot of game-specific "hacks" in the emulator), rather than emulating the RDP itself by sending lower-level commands directly from the RDP to the host GPU with something like Vulkan. This is so-called High-Level Emulation (HLE), and it's a massive shortcut to emulating the whole RDP -- which meant that N64 could even be "emulated" on a PC from 1999.
Lower-level emulation of the RDP itself has been recently made possible, and now it can also now be "up-scaled" to arbitrary resolutions -- instead of just sticking to high level emulation and telling OpenGL or DirectX to render to a larger resolution -- or even worse, scaling the rasterized rendered output frames by treating them as images.
In practice, Mario 64 was just telling the RDP to render triangles anyways so it mapped nicely to OpenGL for the HLE case, but for more "accurate" emulation (this is like getting from 95% to 99%), the RDP itself needs to be emulated as well for things like the Perfect Dark drone camera mentioned elsewhere.