I’ve been using FDB for toy projects for a while. It’s truly rock solid. In my experience it’s the best open source database I’ve used, including mariadb, Postgres and cockroach. That being said, I wish there were more layers as the functionality out of the box is very very limited.
Ideally someone could implement the firestore or dynamodb api on top.
Is basically distributed SQLite backed by FDB. I’ve been scared to use it since I don’t know rust and can’t attest to if mvcc had been implemented correctly.
In using this I actually realized how coupled the storage engine is to the storage system and how few open source projects make the storage engine easily swap-able.
I really wanted to use FoundationDB for building a graph database, but was taken aback by the limitations in record (10+100 kB) and somewhat transaction sizes (10 MB) [1]. And the documentation [2] doesn't really give any answers than "build it yourself."
mvsqlite seems to improve the transaction size [3], which is nice. Does it also improve the key/value limitations?
> Transaction size cannot exceed 10,000,000 bytes of affected data. [---] Keys cannot exceed 10,000 bytes in size. Values cannot exceed 100,000 bytes in size.
Transaction size and duration is limited to keep the latency and throughput of the system manageable under load, from my understanding. It makes sense to some degree even with no background in the design; if you are serving X/rps with a latency of Y milliseconds, using Z resources, and you double Y, you now need to double your resources Z as well, to serve the same amount of clients. You always hit a cap somewhere, so if you want consistent throughput and latency, it's maybe not a bad tradeoff.
mvsqlite fixes the transaction size through its own transaction layer, from my understanding; I don't know how that would impact performance. The 10kb/100Kb key value limit is probably not fixable in any way, but it's not really a huge problem as a user in practice for FDB because you can just shard the value across two keys in a consistent transaction and it's fine. 10 kilobyte keys have pretty much never ever been an issue in my cases either; you can typically just do something like hash a really big key before insert and use that.
To answer myself, it looks like mvsqlite operates as a VFS implementation, which means it works with pages, not rows. That should decouple size limits.
Ideally someone could implement the firestore or dynamodb api on top.
https://github.com/losfair/mvsqlite
Is basically distributed SQLite backed by FDB. I’ve been scared to use it since I don’t know rust and can’t attest to if mvcc had been implemented correctly.
In using this I actually realized how coupled the storage engine is to the storage system and how few open source projects make the storage engine easily swap-able.