I think many people have conjectures, such as this one, but I don't think it's a tech problem, or a "Lisp is too powerful for its own good" problem. It's a "people aren't writing software" problem. History has demonstrated umpteen times that developing large, sophisticated, maintained, and maintainable projects in Lisp is entirely and demonstrably possible. Modern Common Lisp coding practices gravitate toward modular, reusable libraries through proper modules via ASDF packages ("systems") and Common Lisp namespaces ("packages").

The obvious answer as to why people aren't writing software is that almost all of the people able to write good software don't like the language and are writing software in some other language or for some other platform.

I know Lisp enough to have written programs of a few thousand lines in it. I'm not even slightly fazed by functional programming and (tail-)recursion instead of loops. I've read Steele's Common Lisp book from cover to cover. Someone even tried to get me to interview for a job writing Lisp (I politely told them I thought their system could not practically be implemented in Lisp and was, several years and tens of millions of dollars later, eventually proven right).

And I don't think the language has any redeeming features other than garbage collection and documentation, neither of which is notable in 2022. I'm someone familiar with the language who could quickly become productive in any decent Lisp, and that's what I think of Lisp. Can you imagine what a person new to the forest of parentheses, weird identifiers and rejection of 500 years of operator precedence notation thinks?

> I politely told them I thought their system could not practically be implemented in Lisp.

I'm curious to what that was, it sounds like a challenge more than anything.

I'm off a younger generation of programmers, and I've recently found lisp after coming from python. I still adore python and I can understand python's success, but I also adore lisp and its syntax, and I feel like I get the "powerful" argument for it.

For me, for what it's worth, I think the difficulties for lisp to join mainstream is really the thousands of little differences and gotchas that there's no single standard body looking after the users of the language. Princ instead of print, mapcars everywhere, etc. Trivial examples sure, but there's a lot that prevented me from making a few programs without doing through a book.

I'm not saying it's wrong, or it should be such and such a way. I am just simply comparing it to python and pointing to the first big elephant I come across.

Fwiw I also think that clojure is a lang that is capable of capturing more love because it's capable of tying in with Java ecosystem.. but it java is old and slow*; jobs in Java are slowing down, and clojure isn't filling in the gap, kotlin might be.

I take common lisp with me to holidays, because I like to learn it and all the funny words for things, but by far the lisp I use the most in day to day stuff is fennel, which is a lisp that compiles to Lua, and allows me to write plugins for neovim as well as a few other things that uses Lua.

TLDR: yes lisp needs software to be popular but it's not just a case of "make more applications" nor is the problem parents or operator precedence. People like sensibly named keywords.

I personally think things would be a little better if there was a CL to JavaScript transpilier, then you'd start to get some of the js crowd, same with python.

> History has demonstrated umpteen times that developing large, sophisticated, maintained, and maintainable projects in Lisp is entirely and demonstrably possible.

The thing is, developing maintained, maintainable projects is work, and everything I see about Lisp seems to have an undertone of being done, to some degree, for fun. It’s a language that scratches a specific itch; people feel clever about writing elegant code in it.

But the sad truth is that 99% of real application code doesn’t need to be clever or elegant. It needs to be simple and cheap and maintainable by anyone.

the 1% is extremely important though, and the tools for it look very different.

I do algorithm design, a lot of which is seeing what's possible/feasible before committing. I prototyped a classical AI planning system for orchestrating maintenance. there's no battle-tested standard library to reach for here. I needed prolog, I chose picat because it had a good planning engine.

before this, I worked on an NP-hard graph problem that involved tons of topological transformations for heuristics. I wrote the damn thing in C#. it was a nightmare. it's so hard to iterate and experiment in a language like that. even using something like Python/Jupyter, nested loops and horrible messes of bidirectional maps and indices bog you down. I should have done functional. nobody touched my code for two years because it was pure math. I think using C# made it much harder to understand - too much scaffolding obscured the semantics.

now I'm doing another NP-hard graph problem. this time I'm using MiniZinc. because it's concise, because it's powerful enough that I can at least figure out how to specify my problem and see where scale falls down.

so I'm in this position of using a menagerie of bizarre languages not because I'm a hipster obsessed with pretty code, but because my problem domains are really weird and maintainability doesn't matter when you're not sure if what your goal is even possible. something like Lisp - a lingua franca for weird DSLs - would be great, for my very niche use case.

Absolutely true, and it sounds like you're doing really interesting stuff where the language and the way it maps to the problem are more important than the factors I listed above.

Do you stick with the same (maybe-)esoteric language for the production implementation? And if so, does this then drag the rest of the production implementation into that language as well, or do you do the hard bit in one language and the routine stuff in another more common language?

I do the hard bit in the esoteric language, then the glue in a traditional language. for instance, I made a tool that exhaustively steps an implementation through every possible state of a TLA+ model. I wrote "serializers" to parse TLA+ expressions into C# classes and vice-versa, and used this to build a bridge. Parser combinators work great for this stuff, though that's another reason I'd be happier with a Lisp representation for everything.

> people aren't writing software

Maybe not OSS, but somebody is keeping Lispworks and Franz in business.

I'd love to know who. The example apps in Lispworks site aren't commercial successes, as far as I know.

Looks like it's just a small unit in an office park in the tech center in Cambridge: http://www.lispworks.com/contact.html

It doesn't really take many contracts to scrap together office park rent and a staff of a few people. We're talking like < $1 million/year in revenue.

Good on them. Not every company needs to try to be the next Amazon

Already in the 80s the Lisp Machine OS had > 1 MLOC code lines with support for programming in the large. The development environment was networked from the start. A server keeps the definition of things on the network: machines, users, file systems, networks, gateways, printers, databases, mailers, ... The source code and documentation is usually shared in the team and versioned via a common file server.

Nowadays, there are large applications written by groups/teams, which are worked on for three or more decades. For example the ACL2 theorem prover has its roots in the 70s and is used/maintained until today for users in the chip industry.

Common Lisp was especially designed for the development and production use of complex programs. The military at that time wanted to have a single Lisp dialect for those - often applications came with their own Lisp, which made deployment difficult. A standard language for projects was then required.

These were usually developed by teams in the range of 5 - 100 people. Larger teams are rare.

That was not the point. Lisp applications were already a team sport back then. That all or even most Lisp software is written by single person teams is just wrong.

Modern agile development is far more complex than even that: you need entire loosely-coupled teams of developers to be able to seamlessly cooperate with one another, so there are multiple scales of cooperation. Tightening the semantics of interface boundaries is a necessary support for that kind of development and is what's largely meant today by programming "in the large". Highly dynamic "scripting" languages get in the way of this, and that's essentially what Lisp is too.

Lisp isn't essentially a scripting language. Common Lisp was designed as an application language, in rare case it's also a systems language. On a Lisp Machine, for example the whole networking stack was written in it.

Never heard of Javascript?

> The main issue is that Lisp, for all its inherent "power", has very limited tools for enforcing modularity boundaries in code and "programming in the large"

I don't see any mention of "modular" or "boundaries" in the post you linked, so I'm assuming that it doesn't add extra context to your point.

You say "very limited tools for enforcing modularity boundaries", which I'm going to assume means that you believe that Lisps have constructs for creating modularity boundaries (e.g. unexported symbols in Common Lisp), and just don't enforce them (e.g. I can still use an unexported symbol in CL by writing foo::bar), in which case - I don't think that this is actually an issue.

Programmers are capable of doing all kinds of bad things with their code, and shooting themselves in the foot, yet I've never seen an indication that the ability to shoot yourself in the foot with a language noticeably contributes to its popularity (see: C, C++, Perl, Ruby).

Moreover, specializing to Common Lisp, it's not like CL allows you to accidentally access an unexported symbol in a package - you have to either deliberately use :: (which is bad style, and takes more effort than typing :) or you get a hard crash. This is opposed to the above listed languages, which allow you to shoot yourself in the foot in a large number of extremely diverse and interesting manners, often without giving you advance warning - and yet are still far more successful.

------------

I don't believe that the lack of success of Lisps are due to technical deficiencies.

What? How is Lisp (either the family of design decisions or even a specific language/implementation) "inhumane"? What does that even mean?

Inhumane is the wrong word; brutalist is a better word. Brutalist architecture is characterized by minimalist constructions that showcase the bare building materials and structural elements over decorative design. Human's are not logical/mathematical machines and all the irregular and decorative parts of programming languages are often for our benefit even if they're inconsistent, verbose, and limiting.

That's not the experience a Lisp programmer has.

One is not looking at dead code, instead one is working with it interactively.

Lisp is not brutalist concrete, Lisp is wet clay in the programmer's hands.

The main way to program software in Lisp is extending it while it is running and with the development tools as much integrated as possible. Thus a Lisp program has maximum information about itself, there is no separate debug mode, no long build times, no static code.

Lisps are fairly "decorative". It just doesn't combine a pin-striped blazer over a plaid shirt with leopard patterned pants, like many languages.

Here are some traditional notations found in Common Lisp:

  "string"
  symbol
  :keyword-sym
  #:gensym
  package:sym
  `(static content ,inserted ,@spliced)
  (l i s t)
  (im pro per li . st)
  #(v e c t o r)
  6.18e+23
  13/17
  ;; comment
  #*100111101101
  #1=(1 2 3 #1#) ;; circular structure
  #S(point :x 35 :y 44)

Various dialects have various notations. There is syntax highlighting support in editors, too.

Lisp is not just (this (and that)).

And you imagine this is a contraction to what I said?

Well, not exactly. What you said doesn't quite have the form which supports that sort of activity.

> Brutalist architecture is characterized by minimalist constructions that showcase the bare building materials

This definitely doesn't describe the semantics of most Lisps, which operate at the higher levels of abstraction and conceal many implementation and hardware details from you by default.

It also doesn't describe the syntax of most Lisps, including all of the major ones. Common Lisp, in particular, has extensive syntactic sugar, as shown in kazinator's post.

> structural elements over decorative design

I mean, Lisps don't usually go out of their way to look "pretty", but neither do they actively attempt to look ugly. Most Lisps' syntax has had design effort put into it - the authors strive for consistency and ergonomics.

Taste is subjective, and I personally strongly prefer the syntax of Common Lisp to C or Python - both in terms of aesthetic appearance, and in terms of ergonomics.

Furthermore, very few programming languages (excluding most Lisps and all major languages) are intentionally designed to be "decorative". Whitespace, for instance, is added into languages for readability and parsing determinism, not beauty or decorative appeal.

I don't think that this metaphor is accurate at all. There are many criticisms that you can make of both the Lisp family and specifically of Common Lisp, but I don't think that "brutalist" is a valid one.

Why do you think Lisp never gained significant popularity?

I think I understand why certain people really like Lisp -- I have enough experience with enough programming languages to understand the appeal. But I also believe it's unappealing to the majority and I don't think that's a controversial stance. It's not a simple matter of syntax -- I don't think it can be fundamentally still be Lisp and be broadly appealing.

> It's not a simple matter of syntax -- I don't think it can be fundamentally still be Lisp and be broadly appealing.

Python is, semantically, pretty similar to Common Lisp - they have far more in Common than they do with C++, for instance. So, even if it's not the syntax, it's clearly not the semantics or language behavior, either.

I'm pretty sure it's the syntax, though. You used the word "unappealing", which often refers to a surface attraction or draw. I think that this is absolutely correct - Lisps are unpopular in large part to their very weird syntax, and indeed, one of the most common comments that you'll hear when you bring up Lisp is "Is that the weird one with all of the parentheses?"

This is further reinforced by the fact that most programmers have never even tried to run a single line of a Lisp - and the vast majority haven't invested enough time to learn enough to write a non-trivial program with it, and therefore haven't had enough exposure to even judge whether they like the language or not. Therefore, any reason for unpopularity must be superficial.

It's very easy to shallowly dismiss something if

- none of your friends or coworkers use it.

- there are few jobs requiring it.

Basically, you can mock it all you want with practically zero social risk.

Jokes that target people of some specific nationality or regional group are not as tolerated any more nowadays, perhaps depending on the social company, but this is like those jokes. For instance "dumb Polack" jokes.

You've heard the one: "Frederic Chopin and Nicolaus Copernicus walk into a bar, immediately bespying Marie Curie sitting by herself. Stanislaw Lem is tending bar, pouring her a drink. ..."

I think the majority of developers simply don't know whether they would like Lisp or not. It's just not that well known.

Mainly, Lisp was developed on institutional hardware in the era of big iron. It became larger and more complicated, keeping up pace with the capabilities of those machines, which Lisp could easily strain.

Suddenly, microcomputers showed up in the 1970's. The industrial strength Lisps just wouldn't go down from hundreds of kilobytes or megabytes of RAM required down to two-digit kilobytes.

The situation didn't start to improve until probably the late 1980's.

Some developers used Lisp for prototyping, and had the expensive hardware; then to create a product to ship to users on consumer hardware, they rewrote the prototype. That story was exemplified by the CLIPS expert system.

An entire generation of programmers who started on microcomputers in the 1970's and 1980's wouldn't have even had opportunities to use Lisp.

When I started programming in 1982 or so, I inherited a stack of Creative Computing magazine back issues, so I read about all sorts of interesting stuff, including Lisp; but I wouldn't have been able to get my hands on it anywhere: and not for the lack of knowing about it or wanting.

Microcomputers steam-rolled a vast amount of software that preceded them. Software that didn't make the jump from big iron to microcomputers either died or was relegated to niches. Fortran, Cobol, operating systems like Multics, you name it.

We use Unix-like systems today because Unix started on minicomputers that were close in capability to microcomputers. There was only a small lag time between Unix being "bigger iron only" to running on microcomputers. It used pretty modest resources; e.g in 1980, /vmunix was maybe 50-something kilobytes. But even that was a little bit large, and the microcomputer jump did hurt Unix; it allowed disk operating systems like CP/M, PC-DOS and MS-DOS to gain a foothold. Unix never quite recovered; we now have free versions that are quite popular, but mostly in serving and embedded.

Speaking of Unix, I read a book on using Unix in the mid 1980's. As a kid. I had nowhere to actually access it for hands-on experience, but was exciting to read about. You had to be a university student, or else a grown-up working at some company or institution. (Or else, dial into something and break in.)

Anyways, the time lag between the start of the microcomputer revolution, and Lisp being able to run well like on 1970 big iron, was quite long! Long enough to see people retire. There is nobody out there to do mentoring. If millions of programmers started to be interested in Lisp, there would be nobody available for mentoring. Everyone would have to use just books and such. You can hardly find a course to take or anything like that. Some universities use some Lisp or Scheme dialect in a few courses, and that's about it.

Even if every great Lisp hacker who ever lived was still alive and interested in mentoring, it wouldn't be enough; there simply weren't that many people in computing 30, 40, 50 years ago.

Lisp was invented 60 years ago and we're still talking about it today, right now. I don't think the issue is that isn't well known. Plenty of programming languages have sprung to life in that time from nothing and became more popular.

Lisp is more capable now than any time in it's history and if you wanted to learn about it then there's also no better time in history to do that than right now.

Lisp is absolutely not well known. Some programmers may know the word Lisp and that it refers to some kind of programming language, the same way that they might know about Fortran, and possibly Algol, PL/I or Snobol. People talking about it today are not average programmers.

I agree that there is no better time than now, but at the same time, the ratio of Lisp experts to total developers may also be at an all-time low.

But why do you need experts? Plenty of languages have been created out of nothing in the last few decades and have more users than Lisp. Obviously a new language starts with zero experts.

And Lisp has experts! You're literally posting this to a massively popular forum for developers and it's written in a Lisp dialect!

Reddit, arguably one of the largest sites on the Internet, was written in Common Lisp before being re-written in Python.

What do you think it is?

Marketing. Too focused on AI, followed by more than a decade of breathless hype for C++ and Java.

Nailed it. More generally: "non-technical cultural factors".

I would also add some minor incidental technical factors (e.g. early Lisp implementations were far less efficient than C implementation during the time when efficiency really mattered, and then cultural/marketing factors kept C as the dominant language far after the technical balance tilted in favor of Lisp).

agreed, and well said

in broadstrokes, imo, Lisp's endless plasticity of infinitely nested parentheses is both its greatest strength and... its greatest weakness

I love it at Write time. hate it at Read/Maintain time. why I've avoided it for serious work. Python, Java, Go Rust etc are all easier for my brain & eyes to quickly parse on the screen

which is unfortunate. because I still drool at Lisp's linguistic power

Lisp at "read/maintain" time in a good environment is probably the most enjoyable time for me working with Lisp, assuming it's on a code base that followed some sort of coding standards. Introspection, debugging, and navigation in a Lisp environment are incredible.

Still the average function written by a smart lisp programmer is going to make me sigh, take three deep breathes, and set aside some time to focus, while the average function by a smart Go programmer, I can just read.

This readability loss does counteract the ability to reload forms into a running core. Especially now when i am trying so hard to make everything running be immutable and derived from various git repos.

I don't think this is a Lisp thing, though - you get this same problem with Haskell and C++ and Perl as well, because the cultures of all of those languages normalize and encourage writing clever, difficult-to-read code.

Similarly, if you're working by yourself, or in a team with shared norms, you care far less about average function written by a random smart Lisp/Haskell/C++/Perl programmer, and more about what (a) you/your team writes and (b) the ergonomics of the language itself.

In fact, I argue that Lisp is slightly better than the above languages because much of its fanciness comes from macros, which you can trivially expand inline to simplify the code - which you can't do with fancy category theory constructs in Haskell, and is only barely possible with boost:: templates in C++, for instance.

(that isn't to say that I disagree with your point "the average function written by a smart lisp programmer..." - I completely agree with your assessment, I just don't think it's either as big of a problem as it could be, nor is it unique to Lisps)

> . You can see the modern GC-based/"managed" languages, perhaps most notably with Java, as Lisps that avoided this significant pitfall.

An interesting perspective. From my POV, it's hard to think of a less Lisp-like language than Java. COBOL, maybe.