LLMs are Foundational Models, but not all Foundational Models are LLMs.
If you are interested in this space, it does probably make sense to look up what Stable Diffusion (StabilityAI) and Claude (Anthropic) are. Not recognizing them is fine, the pace is pretty incredible in this space and there's lots to learn, but it doesn't make the point you think you are making.
I wasn't attempting to make any point there except not listing out every company they included in their press release.
Not an expert in the space or LLMs, but I'm 100% their target audience: someone technical trying to decipher the release to see if it's of any use to their own company.
LinkedDataHub, a "RDF-native notebook", is not to be confused with LinkedIn DataHub, which is a metadata store/crawler/ui for your data systems: https://datahubproject.io/.
These terms usually show up in the context of networking protocols. Cloudflare has a very quick explainer: https://www.cloudflare.com/learning/network-layer/what-is-th.... To make it even shorter: a control plane is where all the coordination that controls activity (data) happens. The data plane is where the data actually moves around.
Explainers seem to not cover _why_ you would want to separate these "planes". There are several reasons, and I'm no authority, but for starters:
* control messages will have different expectations around them: their amount and frequency, delivery guarantees, urgency with which they are processed. Treating this traffic separately means you can engineer appropriately for data and control traffic.
* last thing you want is the control message "stop processing traffic from IP x.x.x.x port y" to be stuck behind traffic from said IP/port...
In this context, the meaning is somewhat different. They are referring to administrative traffic vs "actual work" traffic. Auth, billing/accounting, configuration updates, that sort of thing. If you are running a SaaS, and your customer is very security conscious and wants none of their precious data to ever leave their VPC, you have 2 options: deploy your software into their VPC completely, making it hard to do a variety of things like upgrades, and increasing complexity; or you can separate control actions from your "worker nodes" and storage, and only deploy the latter into the VPC. You can then work on your control panels, monitor usage, continuously evolve various admin panels and config options, etc, using normal SaaS approaches while the security conscious customer knows that their core data is not leaving their virtual walls and only "bob ran a thing and stored results" goes to the vendor.
This post is about abstracting out common bits of how one implements that, and allowing SaaS offerings to provide that sort of separation easier.
Awesome explanation, thanks! (Particularly "last thing you want is the control message "stop processing traffic from IP x.x.x.x port y" to be stuck behind traffic from said IP/port..")
I should've added, there's an obvious example for the "SaaS control plane" separation, which is equivalent: "stop processing job X that is destabilizing the cluster" should be processed without needing to fight for resources with job X. Same for ACL changes, user deactivations, etc etc. It's generally a good idea to have your control stuff not be subject to whatever instabilities you might be controlling against.
I was in charge of the Twitter data platform team at the time we developed Heron and deprecated Storm. The Mesos component of your retelling is not quite right. Take a look at this comment I wrote around the time we started talking about Heron, addressing the same misconception: https://news.ycombinator.com/item?id=10056479
If he started two years later and there was not a trace of the Prize work at the company, that would be an indicator that the competition was not important. If he started and could still see knock-on effects from the competition, that's an indicator that it was important.
Plus, he didn't just start at Netflix. He "took over the small team that was working and maintaining the rating prediction algorithm that included the first year Progress Prize solution."
Yeah, that sounds like he has some authority on the matter.
Could it not be something more simple like, Netflix didn’t originally have profiles. So my child watching kid shows and me watching action shows were all feeding into the same recommendation system resulting in subpar results. That and they originally had a 5 star rating system which they then dropped.
Its very possible Netflix realized they needed to course correct the UX and as a result the winners algorithm was solving for a problem that no longer applied because it was using assumptions (rating system & no existence of different profiles) that were no longer relevant.
> So basically all research projects have to redirect funds to cover part of the "open access fees"
Yes, the cost of publishing results, as well as review, editing, etc, should be incorporated into the research budget (researchers can of course choose other means of sharing their findings than through publication). In an open access model, the publishing costs are offset by money saved in not paying for access to other articles the research project needs.
I would hope that opting out means that projects have to pay fees on articles they'd otherwise get for free through this agreement, rather than riding on everyone else doing their bit.
One would imagine the fee schedule is set in the agreement UCB reached with Springer (presumably the agreement will be published at some point...)
While HDFS is indeed used for exporting old data and storing some partition mapping metadata, it's clear from the blog post that MetricsDB is much more reliant on BlobStore as well as MetricsDB-specific services.
> The servers checkpoint in-memory data every two hours to durable storage, Blobstore. We are using Blobstore as durable storage so that our process can be run on our shared compute platform with lower management overhead.
I understand your architecture criticism, and think it has merit, but I'm not sure why Apache gets dragged into that. Apache Airflow is in Python. Apache Arrow is in C. CouchDB is Erlang.
There's a ton of projects Apache Foundation hosts that fit your description but it's a mistake, I think, to confuse individual projects with Apache in general. Bad enough that people confuse the license with the foundation.
Sorry, you are right. It would probably be more accurate to use Spring actually. I was thinking of Cassandra/Zookeeper specifically when I said Apache.
Both very technical, both with very little "sell" despite being given by a Camunda co-founder and a Temporal principal eng.
https://www.youtube.com/watch?v=zt9DFMkjkEA "Balancing Choreography and Orchestration" by Bernd Rücker
https://www.youtube.com/watch?v=EaBVzjtSK6A "Building event-driven, reactive applications with Temporal: Workflows vs Sagas" by Dominik Turnow