Or perhaps, just perhaps, the true higher-dimensional move is realizing that choice of programming language isn’t usually the critical factor in whether a project, system, or business succeeds or fails, and that obsessing over the One True Way is a trap.
It might surprise the author to learn that there are many people who:
1) Have tried lisp and clojure
2) Liked their elegance and expressiveness
3) Have read through SICP and done most of the exercises
4) Would still choose plain old boring easy-to-read always-second-best Python for 90% of use-cases (and probably Rust for the last 10%) when building a real business in the real world.
The article could really benefit from some steel-manning. Remove the cute Flatland metaphor and it is effectively arguing that lisp/clojure haven’t been universally adopted because most programmers haven’t Seen The Light in some sort of epiphany of parentheses and macros. The truth is more nuanced.
The reality of modern software development is that most people focus on languages they use for work, and developers are statistically likely to be employed at companies with large numbers of other developers.
The technical merits of languages just aren't relevant to choosing them for most developers, unless they're helping solve a people problem.
"Artisanal" languages like Lisp, and Forth can be fantastic at solving problems elegantly, but that's not the most important thing to optimize for in big organizations where a large portion of your time is spent reading code written by people you've never met who may not have known what they were doing.
Many of the tools that come from big tech are designed to ease the challenges of organizational scale. Golang enforces uniform styles so that you don't have idiosyncratic teams doing their own things. Bazel is a largely language agnostic build system, with amazing build farm support. Apple and Google have both contributed heavily to sanitizers and standard library hardening in order to detect/eliminate issues without reading the code. Facebook has poured vast resources into automatic static analysis. AWS built an entire organization around treating all their internal interfaces the same as external ones.
> "Artisanal" languages like Lisp, and Forth can be fantastic at solving problems elegantly, but that's not the most important thing to optimize for in big organizations ... Many of the tools that come from big tech are designed to ease the challenges of organizational scale.
I think the field of programming languages has grown enough that we have to start acknowledging the future of programming largely won't be in the context of what it means for devs working at large corporations. One of my favorite talks is from Amy J. Ko called A Human View of Programming [1], which argues there are many other ways to look at programming than "tool for generating business activity" and "mathematical construct", which heretofore have been the dominant views of programming languages.
Because there are so many other forms and purposes programming languages can and will take (she goes through them in the talk), so evaluating them and creating them solely on how well they are able to fit into a corporate R&D pipeline is a very narrow and short-term view of the field.
Indeed, it's been the case for a long time now that most people who write programs are not in fact professional software developers. The most used language in the world is Excel, by several orders of magnitude, and it's the opposite of everything devs say a "proper" language must be. There's something we as a field still need to learn from that.
I have very mixed feelings on this topic, starting with how you quantify and weigh something like "most used" for a programming language. To me, the claim feels almost as much a non sequitur as saying the most used building material in the western world is Legos blocks or Play-Doh...
Is the most used bridge-building technique a plank over a small culvert, or the properly engineered bridge that carries constant, multi-lane highway traffic for a century? How do we weigh the usage of resulting products into the usage of a design and production method? Should we consider the number of program users? The users X hours of usage?
Fundamentally, the software field is still just so young and we haven't teased apart the "obvious" different domains and domain rules that we have for production of different material goods. In some sense, the domains and domain rules for material goods emerge out of the connection to culture, economic roles, health, and safety aspects. Whether it falls into civil engineering, building codes, transporation rules, consumer product safety, food and drug, ...
The self-similar way that software can be composed into systems also makes it confusing to categorize. Imagine if we talked about other crafts the same way, and conflated textile manufacturing, clothing design, tailoring, costume making, wardrobe management, scripting, choreography, acting, and dancing as a single field that coordinates the visual movement of fabric on a stage.
> how you quantify and weigh something like "most used" for a programming language.
Define it as # of people who possess the knowledge and resources to effectively use said language to solve a problem they have in their actual lives.
> Fundamentally, the software field is still just so young and we haven't teased apart the "obvious" different domains and domain rules that we have for production of different material goods.
I think we're saying the same thing here from different angles. I said it's developed enough that we can see there are very different ways of doing things. You said it's young enough that we don't know all the different things there are.
As a member of the handmade community, I certainly hope that corporate constraints aren't the main future of the field. I just think it's a major part of the answer as it stands today.
no way? excel? and corporate programmers are not the majority of programmers? -- i mean im a non-corp programmer but i thought i was a special snowflake
Clojure is built on dynamic typing. This is pain. I wrote enough Python (pre-mypy), Javascript, and elisp to say this. Past certain size a dynamically typed codebase becomes needlessly hard to wrangle because of that. Hence the success of Python type annotations and Typescript.
Instead, the world should have seen the light of Hindley-Milner type systems, ML-inspired languages, immutability, or at least not sharing mutable state. Did Haskell fail? Hmm, let's look at Typescript and Rust.
Don't get me wrong, a Lisp is always a great and fun language, and you can write whatever DSL you might like on top of it. But the old joke that "a Lisp programmer knows the value of everything, and the cost of nothing" still has quite a bit of truth to it.
The reason I switched from Scheme to Common Lisp was because I wanted type checking more than I wanted hygienic macros or case-sensitive (by default) symbols.
Plenty of ways to define complex data shapes in Clojure
Spec is definitely underrated here considering it's built into the language and has a wider scope but for most people they want the intellisense experience which you can get with clj-kondo + mailli but is not built in so most teams don't use it, fair enough
I'd like to move the goal posts though and say I want flowstorm in every (any other?!) language
I can just run the program and scrub backwards and forwards through the execution and look at all the immutable values frame by frame with a high level UI with plenty of search/autocomplete options
For program understanding there's nothing better
The fact I can program against the timeline of values of my program and create custom UI on top is crazy
One of the most mind blowing demos to me was Bret Victor's inventing on principle and having a programmable reverse debugger for your language makes those demos viable
I built an emulator recently for work that replays what happens on live locally, combined with flowstorm I can go line by line and tell you exactly what happened and why, no print statements no reruns with my own custom UI customised to our apps interesting parts
This is my appeal to anyone outside of Clojure please build flowstorm for JavaScript and or Python
The design of flowstorm is definitely helped by the fact that 95% of Clojure programs are immutable but I don't think it's impossible to replicate just very difficult
On the other hand, it would be easier to add type checking to a Lisp than it was to Python or JavaScript, and I don’t know any technical reason you couldn’t. A little Googling shows it’s been experimented with several times.
But the real strength of Lisp is in the macros, the metaprogramming system. And I suspect that typing most macros properly would be a bit less trivial than even typing of complex generic types, like lenses. Not typing a macro, and only typechecking the macroexpansion would formally work, but, usability-wise, could be on par with C++ template error reporting.
I think we (the Clojure community) quickly figured out we don't really want static typing, which is a bit evident by the low uptake of Typed Clojure.
Personally I found it to A) make it a hassle for downstream consumers since your design is suddenly impacting others, because you can "lock things down" and B) have that very same effect on your own codebase, where it becomes a lot less flexible where it needs to be flexible.
Nowadays, I just use another language if I want static types, which happens sometimes but not nearly as often to say that dynamically typed languages are "dead" or whatever.
My point was that you could implement type checking with macros, not that you could type check macros. (Though that would be cool!) As opposed to having to change the language definition first (Python) or implement an entirely new compiler (TypeScript).
Certainly you can implement the typechecker with macros, but it should also work on macros, before expansion. That is, you likely want (-> ...) typechecked as written, not (only) as expanded, and typing errors reported on the non-expanded form.
Word. This is a problem of lisps in general, they loose information as the same "thing" traverse the various meta-layers that constitute the system. A parsed expression is not tied to its string, and the expansion of the expression, provided it is a macro, is not tied to the original expression. In the same vein: you can't easily find the source code of a lambda that was compiled/interpreted.
Of course you can do all of this, but you need to build it yourself: see rewrite-clj. If you want to build a clojure debugger that is able to display or refer to code with the same indentation the programmer is dealing with in his text editor, you need to bridge the gap between clojure expressiosn and their string representation.
Anyway I concur that reversible macros would be great. Tag the output, have those tags propagate to the input by playing the macro backwards. Complex stuff really. That's a job for category theory I guess.
If we approach the question as engineers, scientifically, with numbers and studies, not anecdotes and hand-waving, then Clojure is hands down the best language in terms of productivity and bug reduction.
To this day, I know of no study that was able to demonstrate superiority of statically-typed languages - [1].
What studies clearly show, is that both in terms of productivity [2] and bug reduction [3], expressivity reigns supreme.
And Clojure is the most expressive [4] out of languages that can leverage huge ecosystems (Java and JS, soon C++ through Jank dialect).
> Clojure is built on dynamic typing. This is pain. I wrote enough Python (pre-mypy), Javascript, and elisp to say this.
Probably not an absolute truth, but definitely a personal truth for you. For me, it's pretty much the opposite, static/fixed types is such a pain when you just wanna solve a problem and you know how to achieve it, all the invariants/constraints but the language tells you "No, you know what, this other person said you cannot use X for Y, so I'm gonna say no" instead of just letting me do that thing.
With that said, I still reach for Rust for about ~30% of new projects, despite the types, because some languages fit other problems better, simple as that. And still a lot more contracting gigs available for various Rust codebases who've fallen into disrepair, so one does what one can.
I feel like big codebases regardless of their size are hard to wrangle not because of the languages used, but because of the programmers having to rush through building proper abstractions, or even considering not adding so much abstractions. I've seen awful heavily typed codebases as much as I've seen awful dynamically types codebases or awful codebases not using explicit types anywhere, to me there seems to be no correlation between "awful" and "number of explicit types used".
Personally, I prefer a big codebase with lots of (good) unit tests in a dynamic program, than a that same big codebase with no unit tests and explicit static typing everywhere, especially when refactoring and needing to ensure everything (from a business logic perspective) works correctly. But again, this is my personal truth, and I'm not trying to claim it's universal.
"It might surprise the author to learn that there are many people who:
1) Have tried lisp and clojure
2) Liked their elegance and expressiveness
3) Have read through SICP and done most of the exercises
4) Would still choose plain old boring easy-to-read always-second-best Python for 90% of use-cases (and probably Rust for the last 10%) when building a real business in the real world."
This is me to a T — even when I'm building hobby projects. The point of writing any code, for me, is most of all to see a certain idea to fruition, so I choose what will make me most productive getting where I want to go. And while I still worship at the altar of Common Lisp as an incredibly good language, the language matters much less than the libraries, ecosystem, and documentation for productivity (or even effective DSL style abstraction level!), so eventually I have had to make my peace with Python, TypeScript, and Rust.
Tacking on, part of seeing it to fruition, and continued lifetime, is to ensure you can communicate the intent and operation to a large group of potential successors and co-workers.
An incredible epiphany that you can't transmit may not be as useful as a a moderately clever idea you can.
I think the missing piece is that "more expressive" languages do not automatically create more value at the team or company level.
Languages like Lisp, Clojure, Rust, Haskell, Erlang give strong engineers room to build powerful abstractions, but they also increase cognitive load and ramp up cost. In most orgs with churn and constant hiring, you do not get to enjoy "we built great abstractions and now we are fast". You live in "someone new is trying to understand what the last person did".
That is why hand holding and guard rails win. Not because Python or similar are technically superior, but because they support a commoditised, fungible workforce. Even if a wizard in a high dimension language is 2x more productive, that does not necessarily beat a slightly larger team in a mainstream language once you factor in turnover and ramp up. Companies mostly optimise for business impact, predictable delivery, and ease of staffing, not for maximising the ceiling of the top few programmers.
That said, at the individual level, as a programmer, you definitely can benefit from learning and mastering those added dimensions, even if you are to never use them again professionally, they expand your mindset.
> Languages like Lisp, Clojure, Rust, Haskell, Erlang give strong engineers room to build powerful abstractions ... You live in "someone new is trying to understand what the last person did".
In a language like C++, Rust or even Haskell, those "powerful abstractions" are mostly about building libraries, not application code. You still benefit from powerful libraries that were built as part of the ecosystem, even while keeping your own application code simple and intuitive to ease onboarding of new coders. Part of the "powerful abstractions" is to enable a "hand-holding and guard rails" approach for application code that's going to interface with that full-featured library.
There are several languages that I could use and be economically successful with, but I refuse to use because I consider them to be poorly designed.
Using a bad language for 8 hours a day makes me irritable and it's impossible to prevent that irritability from overflowing into my interactions with other people. I'd rather that my conversations with the computer be joyful ones.
Most of the time when someone adds these fancy languages what happens is that they leave and the ones left are the ones that have to deal with the shit that was produced.
I'm going through this now, having to deal with code nobody wants to touch because it is overly complex, has no documentation, and is in a language no one else knows. Now, whenever i see an effort like this, to bring an exoteric language for absolutely no good reason, i try to kill it as fast as possible.
I don't want to be the victim of this code in the future or have my team bear the cost of maintaining stuff they don't understand.
Sadly, much as I love Forth, it's kind of the same thing. It's an awesome language and it's a great way to bring up bare metal to a functional state, but who does that these days?
I could probably include Forth as a scripting language in a bigger app, but that app is probably going to want more complex variables than machine word size ints, and fixed-length strings. So, oh dear, Forth's not a great fit for that, and everyone just uses Lua anyway, so Lua it is.
Which is a pity, because I like Forth, and I used to to create possibly the nerdiest project on Github. I like Forth a lot, and I'd encourage anyone curious about how you get from "chunk of thinking sand and copper" to "thing I can type commands in" to have a crack at it - it's easy enough to implement your own, just to see how it's done.
But I don't expect anyone else to jump up and like it too, just because I said it's cool.
I ported Forth to a 1980s sampler, so that you could plug a MIDI cable in and run a special terminal program to write Forth programs on its 6809 processor.
It boots off a floppy disk, so it was really just a case of working out where in the ROM and OS disassembly the entry points were, and making it all fit around the assumptions the ROM makes.
This allowed me to make up a diagnostics disk to check the RAM (a whopping 128kB of sample RAM) and IO are working.
While what you say is true (I’ve used Lisps for 40 years and here I am writing Rust), the people who consciously make that choice are a tiny niche. There are vastly more people who don’t and can’t make that choice because they don’t have 1-3. So the empirical evidence for what’s actually critical is pretty slim.
yes. and as a long time lisper, i don't think that it's the macros.
i think lisp's magic is a lot more cultural than most people think. i.e. how lispnicks implement lisps and the ecosystem around it. how easy it is to walk the entire ladder of abstractions from machine code to project specific DSL's. how pluggable its parsing pipeline is -- something that is not even exposed in most languages, let alone customizable.
the language, the foundation, of course matters. but i think to a lesser extent than what people think. (hence the trend of trying to hire lispnicks to hard, but non-lisp positions?)
and it's not even an obviously good culture... (just how abrasive common lispers are? need to have a thick skin if you ask a stupid question... or that grumpy, pervasive spirit of the lone wolf...?)
maybe it's just a peculiar filter that gets together peculiar people who think and write code in peculiar ways.
maybe it's not the macros, but the patterns in personality traits of the people who end up at lisp?
> The article could really benefit from some steel-manning. Remove the cute Flatland metaphor and it is effectively arguing that lisp/clojure haven’t been universally adopted because most programmers haven’t Seen The Light in some sort of epiphany of parentheses and macros. The truth is more nuanced.
The talk I posted from Alan Kay is the steel man. I think you've missed the essence of TFA because it's not really about Clojure or lisp.
You may need to explain more? I don’t think I missed the big idea - the metaphor of a separate plane or higher dimension that contains ideas not expressible in the ordinary one is a nice metaphor, and does apply well to some things (Kuhn’s paradigms in history of science come to mind, e.g. Newtonian Mechanics versus Relativity). I just don’t think it really applies well here. What business concepts or thoughts can you express in Clojure that you can’t express in Python or Rust?
Not GP, but … because the overwhelming majority of programming is done in support of businesses selling things?
I’m not just talking about people who program for a living. The majority of academic CS chooses its research directions because of what limits people are running into for business; even privacy-focused software has been commoditized by many business; a large amount of OSS development is by (and often paid for by the employers of) people working for money; heck, after Linus’s initial “just a hobby OS” period even Linux’s contribution base was primarily driven by business needs (even if those needs influenced it via “contributor had a problem at work and committed a solution for it upstream in spare time” as often as “paid contributor upstreamed a change on behalf of their employer”).
Yes and no. Most of the big new languages today are created to support the business of selling things because languages are expensive to make, they don't generate any profit themselves, so the only people who have enough money to fund their development are mega corporations, who act in self-interested ways.
But look at historical languages and why they were created:
Algol - to explore writing algorithms
Fortran - to help scientists write programs using typical math formulas
Matlab - to help write programs in linear algebra
Haskell - to explore lazy program evaluation
ML - to explore how to reason about proof automatically
C - to build an OS
Python - to interface with an OS
LISP - to formalize symbol processing
APL - to explore programs defined over arrays
LOGO - to help young kids to program computers
Prolog - to create a language around the idea of formal logic.
Smalltalk - to create an entire programming system, not just a language
(I've left out C++, Java, and JavaScript because I feel like those languages are mostly about serving business interests)
Pretty much the entire computing landscape over the past 50-70 years has been defined by people writing languages for reasons other than "this is for a business to use to make more money". So if we let business-driven interest dictate the future direction, we will have missed out on all the things that could have been. Would Haskell ever have been invented if businesses interests were the only concern for researchers?
Fortran compilers were historically implemented by hardware vendors in order to sell their hardware, and this still largely holds true across the surviving implementations with the exceptions of GNU Fortran (obviously) and nagfor (commercial s/w product). There's a good reason that Cray Research's software group was initially part of its marketing department.
The post isn't about Clojure or Lisp, it's about the author's journey as a programmer, and the mind-bending effect learning a Lisp had on their development. They're still in the midst of figuring it out, but a lot of people have been on this path before them. TFA has been written over the years by various authors about Prolog, or Haskell, or Smalltalk. In my case I would have written it about Lucid.
The interesting bit here isn't related to Clojure or Lisp, that's what people are chewing on because it's the surface level topic of the essay. The thing that interests me about this post is how it touches on the psychedelic nature of learning programming languages and what that does to one's perspective as a programmer.
So when you ask "what business thoughts can you express in the language", my response is it's not about what you can express, but more about "what new thoughts / ways of thinking has the experience of learning the language caused you to become aware of?".
Few people can go their whole lives writing Python and think all the possible thoughts there are to think about the shapes and forms programming can take. It's hard to develop a good sense for your own practice of programming if you never step outside and see it from other perspectives. It often takes exposure to completely new languages with different design points and abstractions to really give one perspective about their own practice.
The easiest one related to lisp is just the form of the syntax, which is surprising to many students. Most programmers don't even consider you can write (+ 1 1) and that's the same thing as (1 + 1). They don't think about the pros, or the cons, or why one might be better than the other, because every language they've used and ever will use writes it as (1 + 1). But as soon as they see Lisp, they immediately see something that changes their perspective about everything they have previous learned, and therefore will reshape how they approach programming in the future. It doesn't have to mean they will use Lisp going forward, but it does mean they will program in their language of choice with greater purpose. That's how we each hone our craft.
Add on homoiconicity, read/eval, programming as manipulating an AST, and meta programming, and you've got yourself a righteous trip.
Now, can people learn about those things from other sources, without encountering Lisp? Of course. Can you express those ideas in mainstream languages? Yes. But the point is many devs don't think those thoughts until they've had a psychedelic experience, it's very common, that we need to consider that it's part of one's growth journey as a programmer to have these experiences, so we should encourage it. The author of TFA doesn't have experience enough to make that point, as they are on the journey, which is why I brought Alan Kay's talk into context, since his perspective is from the other end of the journey.
That's a bit of an ad feminam attack, isn't it? Just because I used the phrase "business concepts", somehow money is the only thing I care about when it comes to language choice? And yet, in my top-level post I said I went and learned lisp and clojure and read SCIP, and I will add that I did both of those things for fun. So no, I don't only think of programming languages as a way to make money. Elegance and expressiveness are interesting for their own sake. I trained as a mathematician; of course I think that.
But TFA was riffing on Paul Graham's old essay Beating the Averages, which argued precisely that the expressiveness of Lisp gave his startup a business edge. That was the context of my comment. I'd add that most of what most of us do in our day jobs is to use programming languages to make money, and there's no shame in that at all. And if you want to talk about why certain languages get widespread adoption and others not, you have to talk about the corporate context: there is no way around it.
But I'll rephrase my question, just for you: "what abstract problems can you solve or thoughts can you express in Clojure that you can’t express in Python or Rust?"
Oh, I didn’t say GP was making a sexist attack. If I thought that I would have said it clearly and distinctly. Ad feminam is just the feminine version of ad hominem. I like Latin.
I’m sympathetic to looking down on the obsession with money. But there’s something deep and important about the monetary element. Engineering is about solving real-world, practical problems. The cost is a real factor in whether a potential solution is a useful one.
I think the money question is a red herring here. I’d phrase it more like: what problem in a user’s problem space is expressible only like this? And if the only user is the programmer, that’s alright, but feels more aligned with pure academia. That’s important, too! But has a much smaller audience than engineering at large.
some people only think about life as a way to make money. unfortunately coding was best-in-slot career for too long and these kinds of people hijacked the culture.
It might surprise the author to learn that there are many people who:
1) Have tried lisp and clojure
2) Liked their elegance and expressiveness
3) Have read through SICP and done most of the exercises
4) Would still choose plain old boring easy-to-read always-second-best Python for 90% of use-cases (and probably Rust for the last 10%) when building a real business in the real world.
The article could really benefit from some steel-manning. Remove the cute Flatland metaphor and it is effectively arguing that lisp/clojure haven’t been universally adopted because most programmers haven’t Seen The Light in some sort of epiphany of parentheses and macros. The truth is more nuanced.