> One of the coolest parts of our codebase is the new general purpose language at its foundation. Though the semantics of the language are substantially different than Clojure, it’s defined entirely within Clojure using macros to express the differing behavior. It compiles directly to bytecode using the ASM library. The rest of our system is built using both this language and vanilla Clojure, interoperating seamlessly.
JSX would be an (early) example of this class of language. XML is equivalent to s-expressions. So React.js (drunkly) is just a reactive language embedded in a normal language with seamless ability to hop between them. I elaborate about this equivalence here https://www.reddit.com/r/Clojure/comments/mavg81/the_distinc... . The brilliance is they did this so smoothly that nobody noticed, and with dynamic types, and without the typed people even noticing!
Everyone is doing it already, to a large extent. Where "an API" ends and "a programming language" starts is a matter of opinion. It's a fuzzy boundary.
I think it really comes down to how well the new semantics fit the problem domain. If it's a much better fit than the alternative, it's a net win. But just because you can define new semantics, doesn't make it a good idea.
This is Nathan Marz we're talking about though; he's on the short list of people who can both accurately identify the need and create something workable to fill it. If he told me it was necessary for a particular problem domain, I (personally) would think long and hard before I disagreed.
I know people who work at RPL, am 1000% confident that anyone who joins knows _exactly_ what they are getting into (and this kind of crazy Nathan Marz rabbit hole is exactly why they signed up).
I can only guess regarding the use case. I still can make a statement that I personally do not think that the coolest part of a codebase should be a new general purpose language at its foundation. To me this reeks of not invented here syndrome, inner-platform effect and KISS "violations". IMO when you provide new tooling you want your developers be able to stand on the shoulders of giants and let them choose the fights they want to fight. Idiomatic programming is very helpful in this regard. When programming new languages with macros is idiomatic in Clojure so be it, but I doubt that.
I have dabbled in Clojure and Julia, and both of them support macro to make it easier to develop DSL looking syntax. I have always wondered how difficult it it to debug macro heavy code - where's other language cite 'no macro' as a feature such as Zig.
> I have always wondered how difficult it it to debug macro heavy code
It isn't much harder than regular core. `macroexpand' is your friend! Particularly when wrapped by your IDE[0] into a "macrostepper" tool, which lets you macroexpand into an overlay[1], step by step.
Ultimately, macros are just functions - if slightly peculiar ones (they receive unevaluated arguments, and their return values are treated as if they were the code at the point of invocation). You can unit-test them just like any other function.
That's not to say there aren't macros that are very tough to debug. But the problem isn't macro-heavy code, as in code containing a lot of macros. The problem are heavy macros - monstrosities with complex internal logic, expanding their inputs into large amount of code generation, hidden state, etc. Complex macros like these take skill to write[2], but they're also rare to see.
--
[0] - By which I mean Emacs, though I guess there are Clojure IDEs now too.
[1] - I.e. a read-only view replacing the code you're macroexpanding on screen.
[2] - First rule: minimize the amount of code within the actual macro definition, move all logic into functions to be called by the macro, and unit test those extensively.
In my experience with Julia it's no easier or harder than any other code. Macros tend to be pretty transparent, and it's easy to see what code they generate, although I've never had to do so. Most of the time, a library with a macro-based DSL (e.g. JuMP) has a function-based DSL underlying it that's just more verbose.
I kind of like Rust's version: macros are limited and clunky (and really clunky if you want to do anything super fancy), which means they're available, but I only end up using them when they're really necessary (like I'd be copying and pasting huge amounts of code and there's no other way to avoid doing that)
I concur. I think it's nuts when companies do this. There's another well know SaaS that basically invented their own language but I can't recall who. It's the ultimate ego food for the lead engineer imho.
what a company needs is good leadership at every level. not unique tools, and not "10x engineers".
I thought the same. Clojure is already a niche language, but then you go and niche the niche by implementing another language on top.
Could work out if its essential to offering a 10x service, but I doubt the multiplier is that higher to warrant such an engineering cost of developing and maintaining another language on top of Clojure.
Also, there would be many layers to get to the machine code (Custom lang -> Clojure -> Byte code -> JIT -> Machine code), would this impact performance in a way that makes the program too slow and needs re-writing?
Actually this sounds quite horrible.