I think it is important to realize, that we need to understand language on our own terms. The logic of LLMs is not unlike alien technology to us. That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory. But it is wrong to assume that the big progress in linguistics is in vain, the same reason Prolog, Theorem provers, type theory, category theory is in vain, when we have LLMs that can produce everything in C++. We can use the technology of linguistics to ground our knowledge, and in some dark corner of the LLM it might already have integrated this. I think the original divide between the sciences and the humanities might be deeper and more fundamental than we think. We need linguistic as a discipline of the humanities, and maybe huge swaths of Computer Science is just that.
I agree with you. I think the fundamental problem is we don't have a good unified theory of fuzzy reasoning. We have a lot of different formal approaches but they all have flaws.
Now LLMs made a big breakthrough that they showed we can do decent fuzzy reasoning in practice. But at the cost of nobody understanding the underlying process formally.
If we had a good unified (formal) theory of fuzzy reasoning, we could build models that reason better (or at least more predictably). But we won't get a better theory by scaling the existing models, I think Chomsky is right about that.
We lack the goal, not the means. If I am asking LLM a question, what answer do I want? A playfully creative one? A strictly logical one? A pleasingly sycophantic one? A harshly critical one? An out of the box devil's advocate one? A beautiful one? A practical one? We have no clue how to express these modes in logical reasoning.
By way of analogy, the result of the theorem prover is usually actionable (i.e. we can replace one kind of expression with its proven equivalent for some end like optimizing code-size or code-run-time), but mathematicians _still_ endeavor to translate the unwieldy and verbose machine-generated proofs into concise human-readable proofs, because those readable proofs are useful to our understanding of mathematics even long after the "productive action" has been taken.
In a way, this collaboration between the machine and the human is better than what came before, because now productive actions can be taken sooner, and mathematicians do not have to doubt whether they are searching for a proof that exists.
>That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory
As someone who has worked in linguistics, I don't really see what you're talking about. Minimalism is not full of exceptions (please elaborate on a specific example if you have one). Minimalism was created to make the old theory, Government and Binding, simpler.
Yes, and the project can be criticised by reducing until there's no value anymore. Well known instances of this process:
- Predicate Fronting in Free Relatives: In sentences like “What John saw was a surprise,” labeling the fronted predicate is not without problems, Merge doesn’t yield a clear head.
- Optional Verb Movement in Persian: Yes-no questions where verbs can optionally move (e.g., “Did you go?” vs. “You went?”) messes up feature-checking’s binary mode.
- Non-Matching Free Relatives with Pied-Piping: Structures like “In whichever city you live, you’ll find culture” mess up standard labeling, needs extra stipulations.
- Some Subjects in Finnish: Nominative vs. non-nominative subjects (e.g., “Minua kylmä” [me-ACC cold]) complicate that Minimalist case assignment.
but we don't have llms that can "produce everything in c++".
We have LLMs that can get some boilerplate right if you use it in a greenfield project, and will repeatedly mess up your code once it grows enough for you to actually need assistance grokking it.
I teach programming to designers and architects at the local university. We're using Processing quite successfully, because it skips a lot of steps. My daughters are too young and are still doing Scratch (with the great micro:bit). But I think next would be Processing, or Arduino with Micropython. But yes, typing is a problem. My older daughter inputs almost all her text via voice input. At work we're doing a lot of low code for new architectures. I think agentic low code tools for kids would be nice.
I missed out so much. When I was a college student, I rejected photos, as digital cameras where everywhere, and I was a snob, in my high school times taking analog camera photo. I read a lot of books about Zen, living in the moment, but now I regret this. The view photos I have I cherish as very dear memories.
When modern smartphones first came out, I rejected them as a snob too. I continued to believe a dumb phone plus a dedicated camera was a better choice. But of course I often forget to bring the better camera out and I missed out on so much too. There's a reason people say the best camera is the camera you have with you.
15h per month? Even without heavy listening, this sounds really low, doesn't it? And if you want to listen to a series if books, like Malazan book of the Fallen at 388h, you wouldn't know anything about the start by the time you are finished, as it's been two years. I would expect that most casual listeners who only listen to half an hour of content a day instead listen mainly to podcasts?
At least that's what I do when I don't have the time for audiobooks. But maybe I shouldn't generalize from my own experience, as everyone is different.
For me, audible + the occasional torrent works quite well, and I wouldn't like to use the Spotify interface for audiobooks anyway, considering how bad it is for that use case.
Immediately had the same reaction, 15h is less than most of my books runtimes.
I checked out the FAQs on that page and apparently the 15 hours is just included with the Spotify premium subscription and that let's you listen to 15 hours of anything in their catalog.
Separate from that, you can also purchase books outright and it seems listening to those purchased books won't count towards that 15 Hours.
In that context, those 15 hours are actually kinda appealing to me since it would let you listen to more than the 5 min sample of a book you get on Audible before deciding to outright buy it or you could use that time to listen to a couple shorter books every month without having to purchase it.
Think I might have just convinced myself to look into it a bit more lol (I'm curious how good their catalogue is compared to Audibles)
Don't believe them. I was also going to university in Germany and had to work so much to compensate for bad lecturers. Until now I can say I needed 100% of what I learned in university. Even the most esoteric stuff came back to bite me. For LLMs, they are close to useless if you can't review the stuff. Maybe at some point in the future they are better and can reason about their code, but as in fusion, self-driving, etc., you never know when this is. And there will always be people who have to develop this.
To be fair, Java Swing was my first GUI programming experience, and is still the best I had. For Desktop, fast iteration, no budget, and works anywhere, it's basically Swing or Electron.
For me it is only the visiting family use case, where rental doesn't make sense. We have car sharing in the city (Miles) which fixes almost all use cases. Driving to the airport only when without kids, as they both still need car seats, that I can't leave. Otherwise car sharing rental is perfect to get to the airport.
Seriously, with the number of invites, slots, to juggle. Might be better to work again like a dev and with text files in an IDE. Would be funny to send invites with raising a PR. Merging, reviewing, ... what a beautiful mess.
this might be in the way of alan kay's core idea, sending messages with interpreters. if we want to go all the way of the snowcrash pipe dream, then imagining objects with programs thrown around is more realistic than making it somehow work with html. on the other hand, in brings back bad memories of flash silverlight, java applets, that almost destroyed the open web. there is a beauty in easy-to-read text rather than binary blobs.
