Having a ~140 mm tall desktop case under the monitor doesn't work that well with today's pretty large monitors - it would result in uncomfortable viewing angles. Add to that the often very wide monitor stands. And increased cooling noise because the case comes closer to your ears.
Or the case designed for mounting comes with extendable legs to support the weight. Or the monitor is designed on some kind of lever system that can be set to balance. Lots of options.
I kind of like your first idea. The motherboard would be vertically mounted, and then have 90 degree PCIe risers that are structurally reinforced somehow, so that the heavy GPU can also be mounted vertically. You could have 2 of them, one for the GPU and one for additional cards. The whole thing would be mounted via the VESA screw holes on the monitor.
Would probably mostly work for people with quite large monitors though. 40+ inch 4k monitors rock!
... or we just stick with tower cases and some kind of structural reinforcements for the heavy GPU.
