That sounds awesome -- Please post the designs somewhere accessible! My experience of kvms is that they're either cheap and don't work in weird ways or expensive and do work, but with quirks. I think no small part of it is earthing and the avoidance of ground loops at different potentials.
After doing a bunch of research I eventually bought a monitor with a KVM built in and I’ve had almost no issues with it. It does imply if you go this route that you are stuck with a single monitor setup, but if that is how you work anyway it’s totally fine and great.
