When we're talking about x86, that's true in ring 0. Otherwise first thing CPU does is to enter privileged, ring 0 mode, save registers, jump through interrupt vector table and process the trap in kernel code. Trap handler will probably need to check usermode program counter and take a look at the instruction that caused the trap. No hard data, but I think we're talking about 1-5 microseconds.

Runtime/language exceptions have different mechanisms that don't require kernel context switches (but might involve slow steps like stack walk).