So the project I am working on we’ve chosen to use Amazon RDS and Postgres. Speaking with Barak awhile back he suggested that we setup two different cayley instances one for writing one for reading. My newest thought is to do that but take advantage of RDS further by using their Read Replica feature.
We have the Cayley write instance write to the normal RDS and the Cayley Read instance read from the Read Replica, this also will allow us to scale either RDS according to load. The next area I am starting to explore is how vertically scalable the RDS instance will be to the number of quads/concurrent users/query complexity.
We plan on easily having over 1 billion quads stored so the first performance metric will be all single user based simple queries on the 1 billion quads and seeing what size instance we need to handle that. We will than increase complexity of some of the queries, followed by adding more users hitting the database.
This testing should be taking place within the new month as we finish our ingestion engine to cayley. I will be excited to share the results, but looking for any thoughts on the above.