Streaming Cassandra at WePay
I&#x27;m not saying nosql dbs don&#x27;t have their place, but...<p>Articles like this are exactly why my current company moved to cassandra, and why I regret not having tried harder to stop them (but I was new, didn&#x27;t knew cassandra and I&#x27;m digressing)<p>Cassandra is (very probably) wrong for your use case.<p>The only model that makes sense in cassandra is where you insert data, wait for minutes for the propagation, hope it was propagated, then read it, while knowing either the primary&#x2F;clustering key or the time of the last read data (assuming everything has the timestamp in the key, and has been replicated)<p>Anything else in cassandra feels like a hack.
Deleting stuff is kinda bad, it will remain there (marked as tombstone) until the &quot;grace period&quot; has expired (default 10 days). It is really bad unless you put the timestamp in the keys, which you basically must do in cassandra. default 100k tombstones ad your query will be dropped.
You want a big grace period for the data, &#x27;cause if a node goes down that is exactly the maximum time you have to bring it up and hope all data has finished resyncing. Otherwise deleted data will come up again, and your node will have less data. If the grace period has passed or are not confident in the replication speed, absolutely delete that node and add a new one.<p>Bigger grace period means more tombstones (unless you never, ever delete data). Even if all the data&#x2F;tombstones have been replicated to all nodes, the tombstones remain. Running manual compaction once prevents the automatic compaction from running ever again.
You need to pay attention to never scan tombstones, so no queries like select without where, always use a LIMIT or where that will ensure you will not scan deleted data.
(but don&#x27;t use &#x27;where&#x27; on columns that are not keys... Unless X, unless Y on the unless X and so on for everything)<p>They need kafka because the probably need a queue, and there is no way to implement those in semi-realtime in cassandra without the tombstone problem killing your queries.<p>Features have been added and then deprecated (see material views), or very limited in their use (secondary indexes, which basically work only per-partition).
Queries that are not plain select&#x2F;insert (or update, which is the same as insert) have loads of special cases and special cases on the special cases.
LWT are much slower, and are the only transaction&#x2F;consensus you will find. for single-row only.
COPY TO&#x2F;FROM works with csv, but will not properly escape all kinds of data, giving you all kinds of fun. Hell, manual copy-pasting a timestamp from a select to an insert does not work, the format a tiny bit different.<p>Backups&#x2F;restore procedures seem complex and amateurish, especially when you have to restore in an other cluster.<p>You can select how many nodes to query for an operation, but things like &quot;Quorum&quot; are not a transaction or consensus. BATCH is atomic, not isolated. read-after-write works only on exactly the same node (unless you write to ALL nodes, and I have not checked what happens if one is down).
And a sh*tload of other stuff.<p>I could go on, but even if this fits your use case, don&#x27;t use cassandra.
Try scylladb first. Same protocol, missing material views and LWT (until next year apparently), but 10x as fast, probably due to being C++ vs java