My question sounds stupid lol
I just thought of an idea to really make Cayley ultra fast - borrowed from how Redis operates.
Just some background:
Redis’s fame is due to being an ultra fast in-memory database that is used heavily for Caching. They provide some persistence capabilities using the techniques outlined in link above (i.e. taking snapshots at regular intervals etc)
Using a storage engine will only slow Cayley down. Perhaps a really cool option would be to primarily advertise Cayley as an in-memory super-fast graph database with ancillary persistence capabilities (if you enable it) using the strorage engines. The storage engines can be used on a snapshotting basis (or the other techniques Redis uses).
The downside is if server crashes, then loading the database takes longer (but Redis does this already and everyone loves it). There are also consistency issues, but people who use it for this mode will be aware of it and accept the risk. Also the database size is limited by server’s memory - but Redis operates like this too.
Alternatively, Option B is what’s happening right now - you use Cayley with a storage engine with full consistency and persistence.
There is a subtle difference between the 2 options, but differentiating between the 2 options is a selling point.
- Option A: Super-fast in-memory graph database with option to have some persistence
- Option B: Fully persistent graph database.
Some users will want Option A (with a bit of persistence), others would want Option A (with no persistence equivalent to people just using in memory Cayley right now) and others would prefer Option B (equivlanet to using a storage engine).