I’ve found that striping across two drives like the 980 Pro described here or WD SN850 Black drives easily gets direct IO read speeds over 12 GB/s on threadripper pro systems. This assumes a stripe size somewhere around 1 -2 MiB. This means that most reads will not need to be split and high queue depth sequential reads and random reads keep both drives busy. With careful alignment of IOs, performance approaches 2x of one drive’s performance.
IO takes CPU cycles but I’ve not seen evidence that striping impacts that. Memory overhead is minimal, as the stripe to read from is done via simple math from a tiny data structure.
well this really depends on what you use mdraid/zfs and as you said alignment. but only for reads.
if you just use dumb mdraid (without too much optimizations configured) (raid 10) + xfs and read/write on top of it you will end up with quite a high memory + cpu usage. but i/o still will be insanly fast.
I've written this more that it works, but its not a put another drive in and done solution. but if you just want to dumb a second drive into it and use mdraid/zfs you will have an overhead. of course if somebody tunes it and builds the application around it you can trim down the overhead significantly.
