I am not a stakeholder, but I just wanted to congratulate you on keeping the momentum going on the Open-RethinkDB project. I've seen several attempts to restart a stalled/dead OS project in the past, so I realise how much energy it takes to keep the ball rolling after the initial flurry of activity/outrage/frustration etc.
RethinkDB is something we were looking at for a large project. In one way I am glad we didn't choose that platform for it, but in another way, the efforts by the open source community to keep it alive has given me confidence to revisit the platform for future projects.
> RethinkDB is something we were looking at for a large project
From the comments in HN it feels like this was always the case for RethinkDB; folks looking at it, admire it, thinking about using it in their next project but, for one reason or the another, never ending up using it. Which is really a pity.
I don't know about the others, but I am really curious to know why you ended up not picking RethinkDB for your project.
Good question, and one that is worthwhile asking as part of the post mortem process.
In our particular case it is because I (as lead technical person on projects) come from a 30+ year background in SQL. Moving to NoSQL is always a point of trepidation for me purely form a comfort and familiarity angle.
I must say though, in our investigation of NoSQL systems for a couple of projects ReThinkDB (and PouchDB for another mobile project) were the two that seemed to stand out, and seemed to make installation and testing easy, as well as remove a lot of the barriers to dipping a toe in the NoSQL ocean.
So, end of the day it wasn't really a technology issue, but rather came down to familiarity and habits.
NB: For the mobile project, we went with Back& (backand.com) which is essentially a NoSQL wrapper around a MySQL database that we have running behind it. It helped me to get my head around NoSQL while still letting our DBAs build out the data structures in pure SQL. Working well so far.
We were considering RethinkDB for a mobile project, but ended up using Postgres on the backend and Realm in the apps with a custom REST API in between. This worked out really well for us, apart from all the work on implementing our own REST based sync (this was before Realm launched their backend product, which might have made a lot of that work unnecessary).
The reason we ended up with Postgres over RethinkDB was that our data model was very relationship heavy in ways that were easier to represent in a relational database than with RethinkDB's document model.
+1. To expand on my own post above - one of the biggest things that scares me away from NoSQL solutions, apart from the fact that the schema is not 'set in stone' and can be changed anytime by anyone, is the relationship aspect.
Because we build business and ERP extensions most of the time, the requirement for multiple joins across sometimes up to 7 or 8 tables was what I found difficult to manage in NoSQL data stores. Foreign keys, LEFT OUTER JOIN, SELECT DISTINCT, GROUP BY etc. are all second nature to me in SQL, but totally Greek to me in NoSQL.
RethinkDB has joins and I found its grouping to be straightforward and super powerful. Selecting distinct values is as simple as putting .distinct() at the end of your ReQL. While it doesn't have foreign keys, any value can act as one as long as it holds a tables primary key (or other indexed fields), in which case you can do db.table('other_table').get(primary_key_val) (or getAll(foregin_key_val, {index: 'my_index'}) if not the PK, but otherwise indexed).
Granted, it won't validate that the keys exist, however (since its schemaless at the DB level), so nothing will prevent you from inserting a document with foreign keys that are invalid, but from a querying point of view, foreign keys are easy. Would be nice if you could set insert/update-checked constraints though.
You absolutely can set the schema into stone, even when using a nosql database. That 'stone' is in your application code instead of the database though. And even when you use SQL the schema can be changed anytime (by anyone, in practice in most small companies). ALTER TABLE ...
I see your point, and we do tighten security on our database so that only certain users can perform ALTER, DROP etc. No doubt that can be done on NoSQL as well.
I guess I was talking about 'inadvertent' database changes. It seems that things like a simple typo can end up creating new attributes in a table when you didn't want it to, e.g. lets say earlier in my code, I have:
mytable.thisthing = 1
Then later on, someone mis-types:
mytable.thatthing = 2
I now have TWO attributes in my table instead of having the original "thisthing" attribute changed to 2.
An app with an SQL backend will usually throw an 'object/column not found' error if it recognises an attribute/column that is not on the original spec, but a couple of the NoSQL systems I've dealt with will happily take that line of code and work away with it, making debugging down the track a bit of a nightmare.
I am sure you can harden your NoSQL service to prevent this sort of thing, but then it becomes a question of whether the extra effort involved is worth it as compared to sticking with SQL where you are more constrained, but have better control over inadvertent table changes.
I once worked on a system which consisted of about a half-dozen services and MongoDB. At some point we noticed that a large percentage of the documents in a certain collection had somehow had a Date field coerced to String, which caused a bunch of stuff to not work correctly.
A simple typo in your SQL DDL statement can cause 'inadvertent' database changes.
Just like you have your SQL schema defined in a central place, allow only a few select developers to make changes to it, and put any changes under extra scrutiny, you do the same when the schema is defined in your application code.
'.. but what if typos' is not a valid argument. Not in 2016. You can use code reviews, tools (compilers, static type checkers) and good software engineering practices to avoid problems due to typos.
I must admit that I haven't delved deeply into Graph databases at this stage. If I can see a compelling reason that NoSQL/Graph will be a better/easier solution than pure SQL on a project, I am willing to, as an old dog, learn new tricks. ;)
I elaborated on this quite a bit a month ago when it was posted[1], but the cliff's notes version is basically that Postgres + Solr turned out to perform better. It was also more familiar to us. I did actually prototype the project with it, and it affirmed a lot of my warm feelings, but performance was the deal breaker.
I'm really happy to see forward motion on this project though. It's the kind of thing I'd love to be involved in, if I were a completely different person.
After my first project with it, almost two years ago, I've used it in every applicable project since. It's exactly what I've been looking for in a database. Relational Documents, an easy to use query language, with realtime feeds. The fact that it didn't explode on the scene, is depressing. Because it takes all of the strengths from other document/nosql databases, and none of the weaknesses.
We are still using in production and for new projects. It's a very strong product with good engineering, documentation and is open source and have faith it will continue to be a viable option.
It's great to see that getting to an open source foundation is a top priority! As a member of the CNCF's technical oversight committee (and as someone who personally loves RethinkDB), I would love to champion RethinkDB for the CNCF -- but it matters much more to me that it gets to an open source foundation than it is that it lands at the CNCF in particular. RethinkDB team: please reach out to me or to any other CNCF TOC member if we can help!
From an outsider's perspective, CNCF seems like it would be a great fit for a project like Rethinkdb: it looks to have the forward momentum Rethink needs right now, and the proximity to other high profile projects like Kubernetes could encourage adoption.
I am also really curious, and think it could be a valuable asset to the greater startup community. The best tech/product and design doesn't always win.
RethinkDB is something we were looking at for a large project. In one way I am glad we didn't choose that platform for it, but in another way, the efforts by the open source community to keep it alive has given me confidence to revisit the platform for future projects.