One reason for using Ceph instead of other RAID solutions on a single machine is that it supports disk failures more flexibly.
In most RAIDs (including ZFS's, to my knowledge), the set of disks that can fail together is static.
Say you have physical disks A B C D E F; common setup is to group RAID1'd disks into a pool such as `mirror(A, B) + mirror(C, D) + mirror(E, F)`.
With that, if disk A fails, and then later B fails before you replace A, your data is lost.
But with Ceph, and replication `size = 2`, when A fails, Ceph will (almost) immediately redistribute your data so that it has 2 replicas again, across all remaining disks B-F. So then B can fail and you still have your data.
So in Ceph, you give it a pool of disks and tell it to "figure out the replication" iself. Most other systems don't offer that; the human defines a static replication structure.
In most RAIDs (including ZFS's, to my knowledge), the set of disks that can fail together is static.
Say you have physical disks A B C D E F; common setup is to group RAID1'd disks into a pool such as `mirror(A, B) + mirror(C, D) + mirror(E, F)`.
With that, if disk A fails, and then later B fails before you replace A, your data is lost.
But with Ceph, and replication `size = 2`, when A fails, Ceph will (almost) immediately redistribute your data so that it has 2 replicas again, across all remaining disks B-F. So then B can fail and you still have your data.
So in Ceph, you give it a pool of disks and tell it to "figure out the replication" iself. Most other systems don't offer that; the human defines a static replication structure.