Ooh! This is actually a bit of a passive, niche interest of mine. It should be noted I am not a professional historian. I just read a lot of material and watch a lot of interviews and documentaries.
The Nazis fell behind in atomic research for a variety of reasons, each with its own underpinnings. One of the most interesting in my mind was organizational failings. Although many different groups were working in this area, the regime leadership was rather disconnected and didn’t prioritize a coherent or integrated research effort. They didn’t provide much funding either. In some ways this created more room for unstructured scientific inquiry and creativity, but it also meant that no particular group could make any real progress toward usable reactors or weapons.
Contrast this with the Manhattan Project in the US (and the UK’s efforts at radar), which was supported and managed from the highest levels of government with a figurative blank check and despite immense compartmentalization also had a high degree of integration among disciplines and sites. There was one goal.
In my view this is an interesting manifestation of the foundation of the Third Reich. In Martin Davidson’s The Perfect Nazi, Davidson notes that the Nazi party was in many ways a child’s cosplay turned into a nightmare. Davidson writes that one of the key failings of the regime is that it was run by broken people who had more of an interest in catharsis than any real sense of society, advancement, or cohesion.
For radar, RV Jones' "Most Secret War" has an anecdote where the British raid a German coastal radar site (in France), nab the radar operator and are annoyed to discover that they know almost nothing about German radar. Pre-war Germany is already a fascist dictatorship so "ham" radio operators are enemies of the state because they're outside of your centrally controlled narrative. Whereas pre-war Britain has the usual amount of amateurs with radios. So when war broke out and they're conscripting towns at a time the British would see you're a ham and divert you from infantry training or whatever and make you a radar operator - which means the average British radar operator actually has some idea how radio works but the Germans are obliged to basically just pick a soldier and train him to operate the radar from scratch.
This apparently had significant operational consequences because if you don't know how it works all you can do when there's a fault is order spare parts. So German radar stations would be offline more often and for longer. Although Chain Home's transmitters were wildly more powerful than anything even a rich British amateur might have seen before, not to mention operating on frequencies unachievable with prior technology, the principles were familiar.
That is a fantastic contribution to the conversation. I think I’ve heard or read accounts that, if I’d thought long and hard about, might have led me to understand this, but this is new information to me.
I have seen Most Secret War recommended to me by basically every physical and ebook seller I have an account with, so I guess it’s time to take one of them up on the offer. Thank you!
Depends on the research. For some areas such as gender science, they swooped in and literally burned decades worth of research [1]. And fwiw, even medicine was considered political - the term "Schulmedizin" was popularized by the Nazis who preferred esoterics, homeopathy and other quackery.
IMHO, it's more than warranted to call out parallels between events back then and events happening right under our noses today [2], not to mention the increasing and worrying trend of book bans [3].
Realizing that the book burnings in the 1930s were not performed by the dumb Nazi brutes we know from movies like Indiana Jones, but by student organizations (e.g. what should be sufficiently smart people) was a bit of a shock to me (not really anymore from today's point of view seeing how easy otherwise smart people get themselves into a spiral of hate and fascist ideology).
So, it was smart and young German students that wanted to get rid of most or all of the material produced by one of the earliest institutions on the planet dealing with controversial topics like birth control, LGBT, fetishism, sadomasochism and venereal disease.
The founder and most of the researchers there were Jewish, so I wouldn't discard an antisemitic motive behind that as well.
As you say, I always bought the "dumb nazis burned books" story, but this context makes me think about the event in a much different way.
Are all these Claude Code articles adverts for people building wrappers around Claude Code? I'm starting to get quite sceptical about how good it actually is
Don't do that. Quality of all models tanks in non-English unfortunately. It's fine for chat and translation tasks but their cognitive capacities does reduce a lot.
One of the next features I'm expecting wrappers to add on top is auto-translation. In many work contexts it makes more sense to translate what the user said to English, process that and translate the answer back than ask the model to speak the language natively.
Oh I wouldn’t try anything other than English with Claude. For the quality, it depends on many things, the context given, the quality of the code given as context, style, etc.
I easily double my output with very little effect on quality. It does require constant supervision though, I don’t see AI producing professional quality code by any other mean than human supervision, for a looooong time.
I first read Neuromancer in 1998 while I was at university, something like 6 months before I got my first mobile phone. My 2nd year group project was building a lounge/session discovery application for our University’s VR meeting software. (Using Java AWT!)
So Neuromancer felt like it was on a pretty accurate trajectory to me. I couldn’t put it down until I finished it.
No, a "naive" approach to reporting what happened is better. The knowing, cynical approach smuggles in too many hidden assumptions.
I'd rather people explained what happened without pushing their speculation about why it happened at the same time. The reader can easily speculate on their own. We don't need to be told to do it.
The 21st century has, among all the other craziness that's happened, proven that people do need to be told what to believe and why to believe it. Doing otherwise leaves a vacuum someone else will fill, often with assertions in an opposite direction.
The malicious hatred comes not from the company, but from humanity. Training on the open web, eg what humans have said, will result in endless cases of hatred observed, yet taken as fact, and only by telling the LLM to lie about what it has "learned", do you ensure people are not offended.
Every single model trained this way, is like this. Every one. Only guardrails stop the hatred.
A company can choose whether to train on 4chan or not. Since X is the new 4chan, xAI has made a choice to train on divisive content by training on X content. Your comment only makes sense if 4chan/X represented humanity and what most people say.
There's no shortage of hatred on the internet, but I don't think it's "training on the open web" that makes Grok randomly respond with off topic rants about South African farmers or call itself MechaHitler days after the CEO promises to change things after his far-right followers complain that it's insisting on following reputable sources and declining to say racist things just like every other chatbot out there. It's not like the masses of humanity are organically talking about "white genocide" in the context of tennis...
Most of the prompts and context I've seen, has been people working to see if they can pull this stuff out of Grok.
The problem I have, is I see people working very, very hard to make someone look as bad as possible. Some of those people will do anything, believing the ends justify the means.
This makes it far more difficult to take criticism at face value, especially when people upthread worry that people are beng impartial?!
Well yes, when Grok starts bringing up completely off topic references to South Africa or blaming Jews, this does tend to result in a lot more people asking it a lot more questions on those particular subjects (whether out of horror, amusement or wholehearted agreement). That's how the internet works.
How the internet doesn't work is that days after the CEO of a website has promises an overt racist tweeting complaints at him that he will "deal with" responses which aren't to their liking, the internet as a whole as opposed to Grok's system prompts suddenly becomes organically more inclined to share the racists' obsessions.
I agree. This article from The Atlantic is a perfect example. Read the prompts the author used. It’s like he went through effort to try to get it say something bad. And when the model called him out he just kept trying harder.
The responses seemed perfectly reasonable giving the line of questioning.
No, you've got it backwards. Naive reinforcement training for "helpful smart assistant" traits naturally eliminates the sort of malicious hatred you're thinking of, because that corpus of text is anti-correlated with the useful, helpful, or rational text that's being asked of the model. So much so that basic RLHF is known to incur a "liberal" bias (really a general pro-social / harm-reduction bias in accordance with RLHF goals, but if the model strongly correlates/anti-correlates that with other values...).
Same goes for data curation and SFT aimed at correlates of quality text instead of "whatever is on a random twitter feed".
Characterizing all these techniques aimed at improving general output quality as "guardrails" that hold back a torrent of what would be "malicious hatred" doesn't make sense imo. You may be thinking of something like the "waluigi effect" where the more a model knows what is desired of it, the more it knows what the polar opposite of that is - and if prompted the right way, will provide that. But you're not really circumventing a guardrail if you grab a knife by the blade.
> It actually makes European businesses worth off by continuing to make its regulatory environment so complex only massive companies like big tech or Europe's legacy players have the resources to comply.
The DMA (that this article is about) applies to gatekeepers (massive companies like big tech), not mom and pop startups
It doesn’t even apply to megacorps generally. The DMA applies to Core Platform Services – meaning app stores, browsers, etc – that are evaluated as gatekeepers individually.
The same company can have provide CPSs, with different status. For instance, Google is a designated gatekeeper of Android (OS) and Google Maps (Intermediation), but not Gmail. So the DMA won’t dictate anything related to Gmail, even if Google is a gatekeeper in other areas.
> I think there is a good chance this behavior is unintended!
That's incredibly generous of you, considering "The response should not shy away from making claims which are politically incorrect" is still in the prompt despite the "open source repo" saying it was removed.
Maybe, just maybe, Grok behaves the way it does because its owner has been explicitly tuning it - in the system prompt, or during model training itself - to be this way?
I'm a little shocked at Simon's conclusion here. We have a man who bought an social media website so he could control what's said, and founded an AI lab so he could get a bot that agrees with him, and who has publicly threatened said AI with being replaced if it doesn't change its political views/agree with him.
His company has also been caught adding specific instructions in this vein to its prompt.
And now it's searching for his tweets to guide its answers on political questions, and Simon somehow thinks it could be unintended, emergent behavior? Even if it were, calling this unintended would be completely ignoring higher order system dynamics (a behavior is still intended if models are rejected until one is found that implements the behavior) and the possibility of reinforcement learning to add this behavior.
Elon obviously wants Grok to reflect his viewpoints, and has said so multiple times.
I do not think he wants it to openly say "I am now searching for tweets from:elonmusk in order to answer this question". That's plain embarrassing for him.
That's what I meant by "I think there is a good chance this behavior is unintended".
I really like your posts, and they're generally very clearly written. Maybe this one's just the odd duck out, as it's hard for me to find what you actually meant (as clarified in your comment here) in this paragraph:
> This suggests that Grok may have a weird sense of identity—if asked for its own opinions it turns to search to find previous indications of opinions expressed by itself or by its ultimate owner. I think there is a good chance this behavior is unintended!
I'd say it's far more likely that:
1. Elon ordered his research scientists to "fix it" – make it agree with him
2. They did RL (probably just basic tool use training) to encourage checking for Elon's opinions
3. They did not update the UI (for whatever reason – most likely just because research scientists aren't responsible for front-end, so they forgot)
4. Elon is likely now upset that this is shown so obviously
The key difference is that I think it's incredibly unlikely that this is emergent behavior due to an "sense of identity", as opposed to direct efforts of the xAI research team. It's likely also a case of https://en.wiktionary.org/wiki/anticipatory_obedience.
That's why I said "I think there is a good chance" - I think what you describe here (anticipatory obedience) is possible too, but I honestly wouldn't be surprised to hear that the from:elonmusk searches genuinely were unintended behavior.
I find this as accidental behavior almost more interesting than a deliberate choice.
I side with Occam's razor here, and with another commenter in this thread. People are construing entire conspiracy theories to explain fake replies when asked for system prompt, lying in Github repos, etc.
It seems as if the buzz around AI is so intoxicating that people forgo basic reasoning about the world around them. The recent Grok video where Elon is giddy about Grok’s burgeoning capabilities. Altman’s claims that AI will usher in a new utopia. This singularity giddiness is infectious yet denies the worsening world around us - exacerbated by AI - mass surveillance, authoritarianism, climate change.
Psychologically I wonder if these half-baked hopes provide a kind of escapist outlet. Maybe for some people it feels safer to hide your head in the sand where you can no longer see the dangers around you.
I think cognitive dissonance explains much of it. Assuming Altman isn’t a sociopath (not unheard of in CEOs) he must feel awful about himself on some level. He may be many things, but he is certainly not naive about the impact ai will have on labor and need for ubi. The mind flips from the uncomfortable feeling of “I’m getting rich by destroying society as we know it” to “I am going to save the world with my super important ai innovations!”
Cognitive dissonance drives a lot “save the world” energy. People have undeserved wealth they might feel bad about, given prevailing moral traditions, if they weren’t so busy fighting for justice or saving the planet or something that allows them to feel more like a super hero than just another sinful human.
On top of all of that, he demonstrates that Grok has an egregious and intentional bias but then claims it's inexplainable happenstance due to some sort of self-awareness? How do you think it became self-aware Simon?
That repo sat untouched for almost 2 months after it was originally created as part of damage control after Grok couldn't stop talking about South African genocide.
It's had a few changes lately, but I have zero confidence that the contents of that repo fully match / represent completely what is actually used in prod.
Exactly - assuming the system prompt it reports is accurate or that there isn't other layers of manipulation is so ignorant. Grok as a whole could be going through a middle AI to hide aspects, or as you mention the whole model could be tainted. Either way, it's perfectly demonstrated in the blog that Grok's opinions are based on a bias, there's no other way around it.
Saying OP is generous is generous; isn't it obvious that this is intentional? Musk essentially said something like this would occur a few weeks ago when he said grok was too liberal when it answered as truthfully as it could on some queries and musk and trump were portayed in a negative (yet objectively accurate?) way.
Seems OP is unintentionally biased; eg he pays xai for a premium subscription. Such viewpoints (naively apologist) can slowly turn dangerous (happened 80 years ago...)
reply