At the table level, yes, it is an array of groups. The important part is that SIMD compares those small control bytes, not 8 complete keys.
Linear probing with a stride of 8 would work too. But when nearby groups are full, new keys keep moving to the same next empty group. As that group fills up too, the search gets longer. Triangular probing uses larger jumps to reduce this clustering, though it is not faster for every lookup. I added the explanation in the article.
The directory and multiple tables solve a separate problem, growth. A single large array would need all its entries redistributed when it grows. Go splits the storage into smaller tables so growth only rebuilds the affected table, reducing the delay that one insertion can cause. The Go blog explains this motivation.
I've added an overview near the beginning for the big picture, plus explanations of what the different parts help with at the end.
Vertically on a single machine, the two are quite similar, both fan work out across all CPU cores.
The different is on scaling out.
ClickHouse scales by making you describe the cluster yourself. You decide how many shards to split the data into, how many copies (replicas) each shard keeps, which row goes to which shard. The copies are kept in sync by a consensus system ClickHouse Keeper. This is flexible but also more works on operators.
VictoriaLogs takes the opposite bet. When logs come in, the inserter just spreads them across all storage nodes on its own, so there is no sharding key for you to design. When a query runs, the selector asks every storage node in parallel and merges the results. There is no consensus system at all. If you want high availability, you run 2 independent clusters and send your logs to both, rather than having the database copy data internally. So this is simpler and less learning curve. See more here https://victoriametrics.com/blog/victorialogs-architecture-b...
The article could be a bit misleading, as it focuses on the act of one goroutine signaling another. That’s why it emphasizes 'for its signaling to work.'
> "In theory, a condition variable like sync.Cond doesn’t have to be tied to a lock for its signaling to work."
It basically separates signaling from lock management, but not about removing the entire need for a mutex. From a pure technical POV, developers can totally handle locking and unlocking the mutex themselves to protect the process of dealing with shared resources, so it's not about technical limitation.
Including a mutex in sync.Cond and having it automatically unlock the mutex in cond.Wait() is an engineering decision that enforces us, developers, to call Lock() on the mutex beforehand and follow the pattern to avoid panic.
At the table level, yes, it is an array of groups. The important part is that SIMD compares those small control bytes, not 8 complete keys.
Linear probing with a stride of 8 would work too. But when nearby groups are full, new keys keep moving to the same next empty group. As that group fills up too, the search gets longer. Triangular probing uses larger jumps to reduce this clustering, though it is not faster for every lookup. I added the explanation in the article.
The directory and multiple tables solve a separate problem, growth. A single large array would need all its entries redistributed when it grows. Go splits the storage into smaller tables so growth only rebuilds the affected table, reducing the delay that one insertion can cause. The Go blog explains this motivation.
I've added an overview near the beginning for the big picture, plus explanations of what the different parts help with at the end.
reply