Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The idea is that you can tailor the merge functions for the application problem domain, rather than grabbing an off-the-shelf distributed database with subtly different consistency semantics than the one you really need (e.g. Riak, Dynamo and Cassandra are all slightly different in how they reconcile, for very good reasons).

With Irmin, if your your application needs a distributed queue to coordinate workflow tasks, then you grab an MQueue datastructure that explains how it deals with multiple readers and writers, and you use that. If you instead need a distributed set with no strong ordering guarantees, then you can implement this as a series of pull/push/merge operations instead.

One interesting exercise we're doing at the moment is to build the equivalent of Okasaki's purely functional datastructures in Irmin. Since the module signatures of these datastructures are quite similar to their non-distributed counterparts, it should be possible to swap distributed/local datastructures depending on the deployment scenario of the application (with the local ones being much more efficient due to the lack of remote synchronization).

If a datastructure really can't merge something, then it can raise a conflict that will can ripple up to the user interface. The design aims to minimize this by letting the application specify a non-failing merge semantic where possible though.



this is very similar to how Bayou reconciles conflicts

http://research.microsoft.com/apps/pubs/default.aspx?id=7377...

BTW, would be much appreciated if you could point to related work on the subject (papers, other projects, blogs, etc).


That's correct, except that an Irmin client could choose not to reconcile if it would conflict, and just continue on with two active branches (presumably hoping for a future event that would help reconciliation). Bayou's a big inspiration for this system -- there's a filesystem under development that exposes some POSIX semantics using Irmin as a base. It should be possible to build rather interesting datastructures that go beyond conventional filesystems as well, though.

BTW, would be much appreciated if you could point to related work on the subject (papers, other projects, blogs, etc).

That'll certainly happen when we complete the research papers on the subject. It's a little out of scope for a blog post series that primarily focuses on trying to explain the stuff we're building.


Can irmin be used a block device filesystem? Or would it only support object based storage?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: