LISP has been singing homoiconicity as its feature. I lately start to think homoiconicity is really a wart. The macros are a system to program the code, while the code is a system to program the application. They are two different cognition tasks and making them nearly indistinguishable is not ideal.

LISP has a full-featured macro system, thus hands down beats many languages that only possess handicapped macro system or no macro system at all. It uses the same/similar language to achieve it is mere accidental. In fact, I think LISP is an under-powered programming language due to its crudeness. But it's unconstrained macro system allows it compensate the programming part to certain degree. As a result, it is not a popular language and it will never be, but it is sufficiently unique and also extremely simple that it will never die.

What if, we have a standalone general-purpose macro system that can be used with any programming languages, with two syntax layer that programmers can put on different hat to work on either? That's essentially how I designed MyDef. MyDef supports two forms of macros. Inline macros are using `$(name:param)` syntax. Block macros are supported using `$call blockmacroname, params`. Both are syntactically simple to grasp and distinct from hosting languages that programmers can put on different hats to comprehend. The basic macros are just text substitution, but both inline macros and block macros can be extended with (currently my choice) Perl to achieve unconstrained goals. The extension layer can access the context before or after, can set up context for code within or outside, thus achieve what lisp can but using Perl. We can extend the macros using Python or any other language as well, but it is a matter of the extent to access the macro system internals.

Inline macros are scoped, and block macros can define context. These are the two features that I find missing in most macros systems that I can't live without today. Here is an example:

    $(set:A=global scope)
    &call open_context
        print $(A)
    print $(A)

    subcode: open_context
        set-up-context
        $(set:A=inside context)
        BLOCK # placeholder for user code
        destroy-context

> I lately start to think homoiconicity is really a wart.

I sorta agree. The simplicity of the syntax and representation makes it particularly amenable to dynamic modification above basically every other language though. There's a bunch of other properties that make this to case though, such as a very dynamic type system, first-class functions, extremely simple syntax, etc. so it's really a combination of a bunch of factors.

That being said, I think homoiconicity is actually a useful feature, but runtime macro expansion is the dangerous part. What I'd really like to see is a Lisp with very well defined execution orders. That is, macros must be expanded at compile time, and with clear symbols for defining what runs and when. I'm not talking about something like `(macroexpand ...)`, more like `(comptime (my-macro ...))`... `(define (my-func) (my-macro+ 'a 'b 'c))` where it's explicit that a macro executes at compile time, and usage of that macro must be denoted with a special character (`+`, in my example).

I think a general purpose macro language is only as useful as the average code-gen/templating language. It's just string manipulation, which can be harder to reason about than true metaprogramming and might result in some ugly/verbose output and having to context-switch between 2 entirely different languages often. What's particularly powerful about Lisp macros is that usage of them doesn't look any different than a normal application usually, and writing a macro is only marginally different than writing a normal function.

> I think a general purpose macro language is only as useful as the average code-gen/templating language. It's just string manipulation, which can be harder to reason about than true metaprogramming ...

I would like you to reconsider. Predicting a program output is hard. So in order to comprehend a macro programed as code, one need run the macro in their head to predict its output, then they need comprehend that output in order to understand the code. I think that is unreasonable expectation. That's why reasonable usage of meta-programming is close to templating where programmer can reason with the generated code directly from the template. For more higher-powered macros, I argue no one will be able to reason with two-layers at the same time. So what happens is for the programmer to put on his macro hat to comprehend the macro, then simply use a good "vacabulary" (macro name) to encode his comprehension. And when he put his application programming hat, he takes the macro by an ambiguous understanding, as a vocabulary, or some one may call it as a syntax extension. Because we need put on two hats at different time, we don't need homoiconicity to make the two hats to look the same.

Or you do:

``` (require (ast macroexpand))

(display (macroexpand my-macro arg1 arg2)) ``` and call it a day.

My argument isn't that the context-switch isn't there, it's that the context switch will happen regardless and having to think in 2 different languages is more mental overhead than is necessary. I do agree that homoiconicity is not a requirement, but it is nice to be able to do it all in one go and with no additional tooling, editor, etc.. In reality, a sizeable chunk of Lisp programmers are executing their code in a REPL continuously as part of their core development loop, there's no tooling (context) switch, no language context switch, and barely a macro vs application code context switch.

To illustrate, Rust macros are basically that. They have a substantially different syntax to normal Rust code that make it very difficult to quickly grok what is going on. It's a net negative IMO, not a positive.

> To illustrate, Rust macros are basically that. They have a substantially different syntax to normal Rust code that make it very difficult to quickly grok what is going on. It's a net negative IMO, not a positive.

Yeah, more like a syntax extension than macro. But I am saying that you need both. Some time you need powerful macro ability to extend the language. Sometime you just need templating to achieve the expressiveness. With LISP, I get it that you are programming all the time, never templating, right? But I guess you only appreciate templating when you use your macro system as a general purpose system . The benefit of general purpose macro systems is you only learn one tool for all languages, rather than re-learn the individual wheels. And when you judge a language, you no longer bothered by its syntactic warts because you can always fix the expressive part with your macro-layer.

> That being said, I think homoiconicity is actually a useful feature, but runtime macro expansion is the dangerous part.

Practically, why would you ever want a runtime macro expansion?

Well, part of the problem is is passing around quoted forms (ie. `(quote ...)` or in may Lisps, `(...)). If that form contains a macro, and a sizeable chunk of Lisps stdlibs are macros, you probably need to implement runtime macro expansion at least partially. So in order to avoid implementing runtime macro expansion, you need to break the semantics of what a quoted form is, or disallow them entirely. The implementation for the former could get really complicated depending on the cases you want to support, resolving symbols in the corresponding scope comes to mind as being particularly challenging. Removing quoted forms entirely just really isn't an option, it's required for one of the most powerful parts of Lisps in general: built-in syntax templates. So we're back to breaking the semantics of quoted forms, which I think can be done reasonably, if not difficult to implement.

In another word, it (to have runtime macro expansion) is a side effect, a compromise, a wart, rather than a design goal, right?

It can certainly be a design goal if you want it to be. Sometimes truly dynamic programming is what you want and need. I, personally, wouldn't want to write any code like that because I mostly write software for other people that runs while they aren't looking, not for myself that runs when I execute it. Lisps are popular in academic settings where the program behavior is the topic of interest and maximum flexibility is a tool to accelerate progress. These types of scenarios usually have the person who wrote the code directly involved in it's execution, as opposed to service development where you want to be more sure it'll actually work before you deploy it.

Most common case is probably trying to pass `and` to something that applies it to a list, `fold` or `reduce` or similar, and being told some variant of "no deal, and is a macro, not a function" with workarounds like `(reduce (lambda (x y) (and x y)) list))`

Mainly, because it's always run-time. Your program's compile-time is the compiler's run-time. Compilers can be available in a shipped application.

Thanks for posting. A bunch of time ago I was actually looking for MyDef for some reason, but I couldn't remember the exact name and couldn't find it.

You could have a replacement for M4 there.

I made a macro preprocessor in 1999 called MPP which also was able to preserve whitespace in perpetrating multi-line expansions, and so suitable even in indentation-sensitive formats. MPP has scopes and also a namespace system. I didn't develop it further because right around that time, I discovered Lisp and, you know ...

m4 is like that. Lots of templating languages are too. Code generation is useful and roughly what you end up with if the language isn't expressive enough, e.g. C or go. Is yours doing anything different to those?

What the lisp approach gives you is the macro has the parsed representation of the program available to inspect and manipulate. It doesn't have to expand to some fixed text, it can expand into different things based on the context of the call site.

Various languages have decided that syntactically distinguishing macros from functions is a good idea. Maybe the function is foo and a macro would be $foo. For what it's worth I think that's wrong-headed, and the proper fix is to have 'foo' represent some transform on its arguments, where it doesn't matter to the caller whether that is by macro or by function, but it's certainly popular.

M4 only do token-level macros or inline macros. M4 macros are identifier that can't be distinguished from underlying language. M4 macros does not have scopes. M4 does not have context level macros. M4 does not have full programmable ability to extend syntax.

I can define any macro lisp can define, just not with LISP syntax. I do not have a full AST view of the code, due to the its generality that it does not marry to any specific underlying language. But I can have extensions that is tailored to specific language and do understand the syntax. For example, the C extension can check existing functions and inject C functions. MyDef always can query and obtain the entire view of the program, and it is up to the effort in writing extensions to which degree we want macro layer to be able to parse. Embedding a AST parser for a local code block is not that difficult.

It's like the innerHTML thing, for me, I always find the text layer (as string) is more intuitive for me to manipulate than an AST tree. If needed, make an ad-hoc parser in Perl is often simple and sufficient, for me at least.

The ban is not having an issue between China and US. US can do whatever to China, and only need balance potential retaliation. There is no moral debate between countries, just power play.

The ban is an issue between US government/politicians and US people. There are some nominal moral contract between the government and people, and the ban need to be justified and satisfied by people's moral belief, such as open internet. Without sufficient moral justification, people's trust against the government is at risk.

Since you are already familiar with other languages, I suggest just pick a tool that you use daily that is open source in C, and start hacking it.

Some billionaires grow integrity at some point since every thing else is just money. Some billionaires never.

> Indeed. UB in C doesn't mean "and then the program goes off the rails", it means that the entire program execution was meaningless, and no part of the toolchain is obligated to give any guarantees whatsoever if the program is ever executed, from the very first instruction.

This is the greatest sin modern compiler folks committed to abuse C. C as the language never says the compiler can change the code arbitrarily due to an UB statement. It is undefined. Most UB code in C, while not fully defined, has an obvious part of semantics that every one understands. For example, an integer overflow, while not defined on what should be the final value, it is understood that it is an operation of updating a value. It is definitely not, e.g., an assertion on the operand because UB can't happen.

Think about our natural language, which is full of undefined sentences. For example, "I'll lasso the moon for you". A compiler, which is a listener's brain, may not fully understand the sentence and it is perfectly fine to ignore the sentence. But if we interpret an undefined sentence as a license to misinterpret the entire conversation, then no one would dare to speak.

As computing goes beyond arithmetic and the program grows in complexity, I personally believe some amount of fuzziness is the key. This current narrow view from the compiler folks (and somehow gets accepted at large) is really, IMO, a setback in the computing evolution.

> It is definitely not, e.g., an assertion on the operand because UB can't happen.

C specification says a program is ill-formed if any UB happens. So yes, the spec does say that compilers are allowed to assume UB doesn't happen. After all, a program with UB is ill-formed and therefore shouldn't exist!

I think you're conflating "unspecified behavior" and "undefined behavior" - the two have different meanings in the spec.

> C specification says a program is ill-formed if any UB happens. So yes, the spec does say that compilers are allowed to assume UB doesn't happen.

I disagree on the logic from "ill-formed" to "assume it doesn't happen".

> I think you're conflating "unspecified behavior" and "undefined behavior" - the two have different meanings in the spec.

I admit I don't differentiate those two words. I think they are just word-play.

The C standard defines them very differently though:

  undefined behavior
    behavior, upon use of a nonportable or erroneous program
    construct or of erroneous data, for which this International
    Standard imposes no requirements

  unspecified behavior
    use of an unspecified value, or other behavior where this
    International Standard provides two or more possibilities
    and imposes no further requirements on which is chosen in
    any instance

Implementations need not but may obviously assume that undefined behavior does not happen. Assume that however the program behaves if undefined behavior is invoked is how the compiler chose to implement that case.

"Nonportable" is a significant element of this definition. A programmer who intends to compile their C program for one particular processor family might reasonably expect to write code which makes use of the very-much-defined behavior found on that architecture: integer overflow, for example. A C compiler which does the naively obvious thing in this situation would be a useful tool, and many C compilers in the past used to behave this way. Modern C compilers which assume that the programmer will never intentionally write non-portable code are.... less helpful.

> I disagree on the logic from "ill-formed" to "assume it doesn't happen".

Do you feel like elaborating on your reasoning at all? And if you're going to present an argument, it'd be good if you stuck to the spec's definitions of things here. It'll be a lot easier to have a discussion when we're on the same terminology page here (which is why specs exist with definitions!)

> I admit I don't differentiate those two words. I think they are just word-play.

Unfortunately for you, the spec says otherwise. There's a reason there's 2 different phrases here, and both are clearly defined by the spec.

That's the whole point of UB though: the programmer helping the compiler do deduce things. It's too much to expect the compiler to understand your whole program to know a+b doesn't overflow. The programmer might understand it doesn't though. The compiler relies on that understanding.

If you don't want it to rely on it insert a check into the program and tell it what to do if the addition overflows. It's not hard.

Whining about UB is like reading Shakespeare to your dog and complaining it doesn't follow. It's not that smart. You are though. If you want it to check for an overflow or whatever there is a one liner to do it. Just insert it into your code.

> That's the whole point of UB though

No, the whole (entire, exclusive of that) point of undefined behaviour is to allow legitimate compilers to generate sensible and idiomatic code for whichever target architechture they're compiling for. Eg, a pointer dereference can just be `ld r1 [r0]` or `st [r0] r1`, without paying any attention to the possibility that the pointer (r0) might be null, or that there might be memory-mapped IO registers at address zero that a read or write could have catastrophic effects on.

It is not a licence to go actively searching for unrelated things that the compiler can go out of its way to break under the pretense that the standard technically doesn't explicitly prohibit a null pointer dereference from setting the pointer to a non-null (but magically still zero) value.

If you don't want the compiler to optimize that much then turn down the optimization level.

> If you don't want it to rely on it insert a check into the program and tell it what to do if the addition overflows. It's not hard.

Given that even experts routinely fail to write C code that doesn't have UB, available evidence is that it's practically impossible.

> So yes, the spec does say that compilers are allowed to assume UB doesn't happen.

They are allowed to do so, but in practice this choice is not helpful.

On the contrary, it is quite helpful–it is how C optimizers reason.

Nice! If the robots can sustain on solar energy, even better.

That would be great, but I don't think we'll have batteries that can power a tractor for more than a short time on rough and soft terrain anytime soon.

How silly is this.

What prevents linux to achieve the same bandwidth?

Not sure about all other optimisations, but Linux doesn't have support for async sendfile.

It's not about the amount of time kids takes up in your life. It is about how kids changes your life forever. Your priority in life changes, then your life changes. I guess when kids leave us, we need relearn the life.