Genuinely curious if anyone has ideas how an LLM provider could moat AI. They feel interchangeable like bandwidth providers and seem like it will be a never ending game of leapfrog. I thought perhaps we’d end up with a small set of few top players just based on the scale of investment but now I’ve also seen impressive gains from much smaller companies & models, too.
1) Push for regulations around data provenance. If you train on anything, you have to prove that you had the rights to train on it. This would kill all but the largest providers in the USA, though China would still pose a problem. You could work around that bit by making businesses and consumers liable for usage of models that don't have proof of provenance.
2) If you had some secret algorithm that substantially outperformed everyone, you could win if you prevented leakage. This runs into the issue that two people can keep a secret, but three cannot. Eventually it'll leak.
3) Keep costs exceptionally low, sell at cost (or for free), and flood the market with that, which you use to enhance other revenue streams and make it unprofitable for other companies to compete and unappealing as a target for investors. To do this, you have to be a large company with existing revenue streams, highly efficient infrastructure, piles of money to burn while competitors burn through their smaller piles of money, and the ability to get something of value from giving something out for free.
Not if regulation prohibits LLMs from China, which isn't that far fetched to be honest.
I think LLM will turn into a commodity product and if you want to dominate with a commodity product, you need to provide twice the value at half the cost. Open AI will need a breakthrough in reliability and/or inference cost to really create a moat.
Not prohibit training, but make it illegal for US companies to embed, use or distribute LLMs created by China. Basically the general consumer will need to go through the effort of use the LLM that they want, which we know means only a small fraction of people.
That assumes that only the US is interested in using LLMs commercially, which isn't really true. Even if you can get America to sanction Chinese LLM use, you aren't even going to American allies to go along with that, let alone everyone else.
China's biggest challenge ATM is that they do not yet economically produce the GPUs and RAM needed to train and use big models. They are still behind in semiconductors, maybe 10 or 20 years (they can make fast chips, they can make cheap chips, they can't yet make fast cheap chips).
Canada would be reluctant. So would Europe, South Asia and so forth. The biggest hurdle for China is the CCP. It is one thing to use physical products from China but relying on the CCP for knowledge may be a step too far for many nations.
Most people won't care if the products are useful. Chinese EVs, Chinese HSR, Chinese industrial robots, Chinese power tech, they are already selling. LLM isn't just a chatbot, it could be a medical device to help in areas without sufficiently trained doctors, for example.
Most people may not care but that doesn't matter if the government cares. I will not be surprised if countries restrict LLMs to trusted countries in the future. Unless there is a regime change in China, seeing it adopted in other countries may be an issue.
It is a very large world though, and nationalism is more of an American shtick at the moment. It is totally possible that countries have to decide to trade with China or the USA (if they put down an infective embargo), but then it really depends on what America offers vs. China, and I don't think that is a great proposition for us.
Nationalism is not merely having having a moment as an "American shtick", though I don't know much about how widespread it is in non-Western developing countries. Certainly might not be so much, there.
The real possibility exists that it would be better to be an independent 'second place' technology center (or third place, etc) than a pure-consumer of someone else's integrate tech stack.
China decided that a long time ago for the consumer web. Europe is considering similar things more than ever before. The US is considering it with TikTok.
It's not hard to see that expanding. It's hard to claim that forcing local development of tech was a failure for China.
Short of a breakthrough that means the everyday person no longer has to work, why would I rather have a "better" but wholly-foreign-country-owned, not-contributing-anything-to-the-economy-I-participate-in-daily LLM or image generator or what-have-you vs a locally-sourced "good enough" one?
We are definitely heading into untreaded territory. It is one thing to use Chinese EVs, but using a knowledge system that will be censored (not that other countries won't be censored) and trained in a way that may not align with a nation's beliefs is a whole different matter.
> using a knowledge system that will be censored (not that other countries won't be censored) and trained in a way that may not align with a nation's beliefs is a whole different matter.
this is the exact same story told to the public when Google was kicked out of China. you are just 15 years late for the party.
> They are still behind in semiconductors, maybe 10 or 20 years
i dont believe they're as behind as many analysis deems. In fact, making it illegal to export western chips to china only serves to cause the mother of all inventions, necessity, to pressure harder and make it work.
They will definitely throw more resources at it, but without even older equipment from the west, they have a bigger hill to climb as well. There are lots of material engineering secrets that they have to uncover before they get there, so that’s just my estimate of what they need to do it. I definitely could be wrong though, we’ll see.
> China's biggest challenge ATM is that they do not yet economically produce the GPUs and RAM needed to train and use big models. They are still behind in semiconductors, maybe 10 or 20 years (they can make fast chips, they can make cheap chips, they can't yet make fast cheap chips).
You don't need the most efficient chips to train LLMs. those much slower chips (e.g. those made by Huawei) will probably take longer for training and they waste more electricity and space. but so what?
because Chinese is investing heavily on all sorts of renewable energies. its annual investment is more than the total US and EU amount combined.
"China is set to account for the largest share of clean energy investment in 2024 with an estimated $675 billion, while Europe is set to account for $370 billion and the United States $315 billion."
Making something illegal isn't gonna work if there's clearly value in doing it, and isn't actually harmful. Regulatory unfairness is easily discerned.
Look at how competitive chinese EVs are, and no amount of tarriffs are gonna stop them from dominating the market - even if americans prevent their own market from being dominated, all of their allies will not be able to stop their own.
Like I've said before, this isn't a physical good that is being sold. LLM is knowledge and many governments and people are going to be concerned with how and who is packaging it. Using LLMs created by China will mean certain events in history will be omitted (which will not be exclusive to China). How the LLM responds will be dictated by the LLM training and so forth.
LLMs will become the ultimate propaganda tool (if they aren't already), and I don't see why governments wouldn't want to have full control over them.
In the early days of Google, people believed there could be absolutely no moat in search because competition was just "one click away" and even Google believed this and deeply internalized this into their culture of technological dominance as the only path to survival.
At the beginning of ride sharing, people believed there was absolutely no geographical moat and all riders were just one cheaper ride from switching so better capitalized incumbents could just win a new area by showering the city with discounts. It took Uber billions of dollars to figure out the moats were actually nigh insurmountable as a challenger brand in many countries.
Honestly, with AI, I just instinctively reach for ChatGPT and haven't even bothered trying with any of the others because the results I get from OAI are "good enough". If enough other people are like me, OAI gets order of magnitudes more query volume than the other general purpose LLMs and they can use that data to tweak their algorithms better than anyone else.
Also, current LLMs, the long term user experience is pretty similar to the first time user experience but that seems set to change in the next few generations. I want my LLM over time to understand the style I prefer to be communicated in, learn what media I'm consuming so it knows which references I understand vs those I don't, etc. Getting a brand new LLM familiar enough to me to feel like a long established LLM might be an arduous enough task that people rarely switch.
The problem with ChartGPT is that that dont own any platform. Which means out of the 3 Billion Android + Chrome OS User, and ~1.5B iOS + Mac. They have zero. There only partner is Microsoft with 1.5B Window PC. Considering a lot of people only do work on Windows PC I would argue that personalisation comes from Smartphone more so than PC. Which means Apple and Google holds the key.
It is really unbelievable how much money companies will spend to avoid talking to and thoroughly understanding their users. OpenAI could probably learn a lot from interviewing 200 random people every 6 months and seeing what they use and why, but my guess is they would consider that frivolous.
There's still one thing missing here: the browser. I do not agree with Gruber's analogy that the LLM is the browser. The interface to the LLM is the browser. We have seen some attempts at creating good browsers for LLM but we do not have NetScape, IE/Edge, Chrome, FF, Brave yet. Once we do, you would very easily be jumping between these models, and even letting the browser pick a model for you based on the type of question.
Also companies will be (and are) bundling these subscriptions for you, like Raycast AI, where you pay one monthly sum and get access to «all major models».
They all work the same and each has its own pro's and con's for each model they launch. Even the API's are generic. It's a bit more difficult to lock in 3rd party partners using your API if your API literally is English. It's going to be a race to the bottom where the value in LLMs is the underlying value of the GPU-time they run on plus a few % upmark.
First, get government regulation on your side. OpenAI has already looked for this, including Sam Altman testifying to Congress about the dangers of AI, but didn't get the regulations that they wanted.
Second, put the cost of competing out of reach. Build a large enough and good enough model that nobody else can afford to build a competitor. Unfortunately a few big competitors keep on spending similar sums. And much cheaper sums are good enough for many purposes.
Third, get a new idea that isn't public. For instance one on how to better handle complex goal directed behavior. OpenAI has been trying, but have failed to come up with the right bright idea.
One view is that it’s not first mover, but first arriver advantage. Whoever gets to AgI (the fabled city of gold or silver?, Ag pun intended) will achieve exponential gains from that point forward, and that serves as the moat in the limit. So you can think of it as buying a delayed moat, with an option price equivalent to the investment required until you get to that point in time. Either you believe in that view or you don’t. It’s more of an emotional / philosophical investment thesis, with a low probability of occurrence, but with a massive expected value.Meanwhile, consumers and the world benefit.
What if the AGI takes an entire data center to process a few tokens per second. Is the still a first-arriver advantage? Seems like the first to make it cheaper than an equivalent-cost employee (fully loaded incl hiring and training) will begin to see advantage.
What if the next one to get there produces a similar service for 5% less? Race to the bottom.
And would AI that is tied to some interface that provides lock-in even be qualified to be called general? I have trouble pointing my finger on it, but AGI and lock-in causes a strong dissonance in my brain. Would AGI perhaps strictly imply commodity? (assuming that more than one supplier exists)
Depending on how powerful your model is, a few tokens per second per data center would still be extraordinarily valuable. It's not out of the realm of possibility that a next generation super intelligence could be trained with a couple hundred lines of pytorch. If that's the case, a couple tokens per second per data center is a steal.
Good point. It’s 2 conditions and both have to be true :
- Arrive first
- Use that first arrival to innovate with your new AGI pet / overlord to stay exponentially ahead
Exponential gains from AGI requires recursive self improvement and the compute headroom to realize them. It's unclear if current LLM architectures make either of those possible.
People need to stop talking about "exponential" gains; these models don't even have the ability to improve themselves, let alone at this or that rate. And who wants them to be able to train themselves while being connected to the Internet anyway? I sure don't. All it takes for major disruption is superhuman ability at subhuman prices.
What does AGI even mean in this case? If progress toward more capable and more cost-effective agents is incremental, I don't see a defensible moat. (You can maintain a moat given continued outpaced investment, but following remains more cost-effective)
Since we're talking about the economic impact here, AGI(X) could be defined as being able to do X% of white collar jobs independently with about as much oversight as a human worker would need.
The exponential gains would come from increasing penetration into existing labor forces and industrial applications. The first arriver would have an advantage in being the first to be profitably and practically applicable to whatever domain it's used in.
Why would the gains be exponential? Assume that X "first arrival" develops a model with a certain rnd investment, and Y arrives next year with investment that's an order of magnitude less costly by following, and there's a simple enough switchover for customers. That's what's meant by no defensible moat; a counterexample is Google up to 2022 where for more than a decade nothing else came close in value prop. Maybe X now has an even better model with more investment, but Y is good enough and can charge way less even if their models are less cost-effective.
> ... Google up to 2022 where for more than a decade nothing else came close in value prop. Maybe X now has an even better model with more investment ...
I was very confused at this point because I haven't really seen X as a competitor to Google's ad business, at least not in investment and value prop... Then I saw you were using X as a variable...
Yes, "first to arrive at AGI" could indeed become a moat, if OpenAI can get there before the clock runs out. In fact, that's what's driving all the spending.
Regulatory capture. Persuade governments that AI is so dangerous that it must be regulated, then help write those regulations to ensure no one else can follow you to the ladder.
That's half the point of OpenAI's game of pretending each new thing they make is too dangerous to release. It's half directed at investors to build hype, half at government officials to build fear.
One way to build a technical moat is to build services which encourage lock-in, and therefore make it hard to switch to a competitor. Some of OpenAI's product releases help facilitate that: the Assistant API creates persistent threads that cannot be exported, and their File Search APIs build vector stores that cannot be exported.
Genuinely curious if anyone has ideas how an LLM provider could moat AI.
Patents. OpenAI already has a head start in the game of filing patents with obvious (to everybody except USPTO examiners), hard-to-avoid claims. E.g.: https://patents.google.com/patent/US12008341B2
> curious if anyone has ideas how an LLM provider could moat AI
By knowing a lot about me, like the details of my relationships, my interests, my work. The LLM would then be able to be better function than the other LLMs. OpenAI already made steps in that direction by learning facts about you.
By offering services only possible by integrating with other industries, like restaurants, banks, etc... This take years to do, and other companies will take years to catch up, especially if you setup exclusivity clauses. There's lots of ways to slow down your competitors when you are the first to do something.
Create closed source models that are much better than the other ones, don't publish your tricks to obtaining your results, and then lock them down in a box that can't be tampered with? I hope it doesn't go that way.
Alternatively, a model that takes a year and the output of a nuclear power plant to train (and then you can tell them about your tricks, since they aren't very reproducible).
An algorithmic breakthrough IMHO. If someone finds out either how to get 10x performance per parameter or how to have a model actually model real causality (to some degree) they will have a moat.
Also, I suspect that the next breakthrough will be kept under wraps and no papers will be published explaining it.
The people working on it will still be allowed to move companies and people talk to each other informally. These Chinese groups working on this with far fewer GPUs appear to be getting 10x results tbh. Maybe they have more GPUs than claimed but we’ll see.
