I just finished this book. I thought it was a great read, capturing most of the problems (with product, sales, marketing, competition) Nvidia faced in its life and how exactly the team solved them each and every time. It also goes into how the management, org structure, and culture have empowered its engineers to find creative solutions to staying alive in the brutal graphic chips industry and continuously discover and exploit new market opportunities.
I learned a lot about how important not just superior technology, but better operations, marketing, sales, and culture are all critical to a successful business.
The only con of this book is that it skips over some parts of nvidia’s history like the short-lived crypto boom, failed acquisition of ARM, etc. It’s still just a minor flaw in an otherwise great book though.
You missed Finance. There is lot of financial engineering going on to keep orgs alive and quickly adapt to ever changing needs. Dealing with cross border diff in taxes, subsidies, tariffs, regs, forex, interest rates, real estate/rent, labor costs and their constant changes is no joke.
Each of these specializations (just like tech) are evolving at their own rates and are all equally capable of outpacing each other depending on the environment.
I didn't. In my worldview, I put Finance as the majority stakeholder in "operating" the company even though there are plenty of teams and roles with somethingOps in their titles that might disagree.
The tech part of tech companies is the easy part in my opinion. Step out into these other domains where you have to deal with _people_ and a company's skill or lack thereof gets really obvious really quick.
And, god forbid, meta-issues like long-term cashflow planning when you're making big changes to your internal organizations and processes, which lets you worry less about funding. And/but investing in complex thinking about operations is part of the book's thesis.
Does it mention the investment from sega that saved Nvidia from going out of business. I think the Nvidia chips became part of some sega arcade boards, might be wrong about that.
That's at the beginning of the book, and it was apparently something of a debacle. The NV2 chip that was supposed to go into Sega's next-gen console never made it, but a $5M investment from Sega did keep Nvidia afloat until the Riva128 launched.
Despite buying a Dreamcast on launch day I only heard about this story in the last week or so ago in a YouTube comment with Jensen Huang congratulating sega on the new virtua fighter. Unfortunately he keeps calling it virtual fighter:
The whole ARM acquisition attempt really smelt of subtle stock market games, because the alleged benefits of such a deal made no sense. With the Arm CEO being ex Nvidia I'm sure he was in on it. Looking forward to someone writing up an account of what really happened.
I think that the blocking of the deal by competition authorities shows the potential benefits of the deal (for Nvidia) in having control of Arm, in addition to making Arm-based SoCs, were real.
Also, Rene Haas (ex Nvidia) became CEO of Arm after the deal fell through. Simon Segars was CEO when the deal was announced.
> I think that the blocking of the deal by competition authorities shows the potential benefits of the deal (for Nvidia) in having control of Arm, in addition to making Arm-based SoCs, were real.
China was concerned about more US influence. The other regions were concerned about stifling of innovation. In reality, a successful acquisition would have meant that most customers would have started transitioning away from ARM and sooner to available RISC-V or Cadence/Synopsis solutions.
> Also, Rene Haas (ex Nvidia) became CEO of Arm after the deal fell through. Simon Segars was CEO when the deal was announced.
Rene Haas was EVP of the whole IP business (CPUs, GPUs, NPUs, etc) before becoming CEO. His influence with the CEO can't be understated.
I posit that this was an enormous gamble to boost valuations of both NVIDIA and ARM and that they knew from the beginning that the deal would not go through. Anyone who understands the ARM business model could have told you that.
> I posit that this was an enormous gamble to boost valuations of both NVIDIA and ARM and that they knew from the beginning that the deal would not go through.
You posit it without saying why the valuations would go up given that you think they - and everyone else given what you say - thought the deal wouldn’t complete.
I'm very glad I read the book, but it did lean a little towards the hagiographic side. From the outside it looks like the same high-pressure culture that drives so much innovation at Nvidia also drove their conflict with Apple.
+1 with your assessment of trending towards a hagiography. The author is not Isaacson. I wish he had spent more time with the people that probably don’t have the best view of Huang’s methods, and approach to running the company. Showing the other side in a more vivid way so the reader can make their own conclusion would strengthen the book and provide a holistic, fuller profile of Huang and Nvidia. Probably the price for access, and still better than nothing…
Nvidia appears to be another pressure-cooker, dog-eat-dog SV company, that outlasted competitors thanks to shrewd bets and urgent execution. Clearly, a great accomplishment, but we’ve seen other companies succeed with similar culture… this isn’t a “new way”… a relentless, obsessive, ruthless technical founder with good business sense (Huang like Gates were able to secure partnerships/licenses of needed technology early on.. same mold as Musk, Zuck) who understands the technology and business implications in detail is a movie we’ve seen before..
I sigh a little whenever anyone mentions Isaacson. My book had a different goal. I wanted to write a serious, substantive business history of Nvidia and explore the reasons for its success. Walt has a different style and focus, which wasn't what I aimed to do.
Thanks for the kind words swimwiththebeat. I really appreciate it. I didn't get to the ARM acquisition failure and crypto boom given time constraints. There is so much to cover in Nvidia's three decade history, so I had to pick and choose.
A card released in September 1997 is described as "an aging and slowly dying chipset." I suppose it parallels AI today, where a model released a year ago is already obsolete on the high end...
Nostalgia for the 1990s is real. I loved that era. PC gaming was awesome. I bought tons of 3D graphics cards from Rendition Verite, 3dfx Voodoo and obviously Nvidia. Having John Carmack tweet about my book nearly made me faint (surreal!). I adored his Doom and Quake games so much.
One of the reasons why Nvidia managed to get ahead was that they managed to get a new chip out every 6 months without fail, so there was always a new Nvidia card to upgrade to.
It's not, and the "Talk to Sales" and the ToS (https://www.dalmatian.ai/terms) strongly suggest they're targeting enterprise customers with bespoke enterprise-y pricing.
Right, we are not open source, but the IDE is free to use. The Slack integration and other product offerings that are in the works will be in the 'premium' version of the product that's sold to enterprise
She definitely knows, she’s just trying to avoid any chance of future litigation by feigning ignorance. Makes sense since OpenAI’s been getting a lot of bad press for using copyright data in training their models.
If someone asks an executive whether they do something illegal, and they reply that they don’t know; does it really protect them from anything or does it simply show negligence?
It protects the company, admission is pretty much asking for a law suite.
For these companies it’s especially important since even if copyright claims will come down the line they need to become big enough to be able to push back on them so they could force a favorable settlement.
Calling it "lot of bad press" seems like an overstatement (maybe looking through hn tinted glasses). Overall, I do not get the impression that people care super much relativ to the overall public interest in everything OpenAI.
I would be shocked if there weren't, being a company with over hundred millions users, that is in the process of reshaping the world (for better or worse).
Being sued in general is not that great of an indicator for anything, in a world filled with lawyers and angry people.
As someone who wants to learn and explore more areas of computer science and programming, this outline and centralized list of resources is incredibly helpful! Thanks so much.
I agree, I don't think the purpose of sci-fi is to predict the future. The future is just impossible to predict due to the myriad factors, variables, unknown unknowns, and second-order+ degree effects an action can have.
The purpose of sci-fi IMO is moreso to:
1. Provide an entertaining story/narrative with technology as the main focus of the world and characters' actions
2. Define a set of concepts to help you think about technology and its possible effects on humans and the world
3. Nudge people to think about what kind of future they would want or not want and how they can use or control technology to achieve that
I do think predictions are an unavoidable side-effect of writing science fiction, but those predictions are almost always in service to those purposes you listed. Stephenson must think nanotech book AI is more plausible as a form idealized childhood education than say, magic fairies. But in the end he relies on our suspension of disbelief to get on past that to the big questions of nurture vs nature and the quality of human intelligence.
1. First of all, thanks for outlining how you trained the model here in the repo: https://github.com/NumbersStationAI/DuckDB-NSQL?tab=readme-o...! I did not know about `sqlglot`, that's a pretty cool lib. Which part of the project was the most challenging or time-consuming: generating the training data, the actual training, or testing? How did you iterate, improve, and test the model?
2. How would you suggest using this model effectively if we have custom data in our DBs? For example, we might have a column called `purpose` that's a custom defined enum (i.e. not a very well-known concept outside of our business). Currently, we've fed it in as context by defining all the possible values it can have. Do you have any other recs on how to tune our prompts so that this model is just as effective with our own custom data?
3. Similar to above, do you know you can use the same model to work effectively on tens or even hundreds of tables? I've used multiple question-SQL example pairs as context, but I've found that I need 15-20 for it to be effective for even one table, let alone tens of tables.
Hi, Till here, worked on the DuckDB-NSQL model on MotherDuck side.
1. definitely training data (for me), we explored about 10 different directions before settling on the current approach. It's easy to underestimate the effect of training data on the quality of the model. Starting point was the benchmark dataset though, which we assembled manually (to avoid data pollution and also because there was simply no text2sql benchmark that covers anything else than plain old SQL select statements with a handful of aggregate functions). And training is also not a one-off thing. With large datasets it is hard to evaluate the quality of the dataset without actually training a few epochs on it and run the benchmark.
3. No way - I see a common stack emerging (take a look at companies like https://vanna.ai/, https://www.dataherald.com/, or https://www.waii.ai) that is mainly centered around foundation models like GPT-4 with strong in-context learning capabilities (that's a kind of a must to make these approaches work and comes with long inference times and higher costs). These solutions include things like embedding-based schema filtering, options for users to enrich metadata about tables and columns, including previous related queries into the context etc. around the model. I'd say it's a bit of a different problem from what we aimed at solving.
I didn't see this in the blog post, but did you train this from scratch or finetune an existing base model?
If from scratch, quite impressive that the model is capable of understanding natural language prompts (English presumably) from such a small, targeted training set.
I see so many business leaders touting the promise of LLMs allowing business to "talk" to their data. The promise does sound enticing, but it's actually kind of hard to get working in practice.
A lot of our databases at work have columns with custom types and enums, and getting the LLM (Llama2) to write SQL queries to robustly answer natural language questions about the data is tough. It requires a lot of instruction prompting, context, and question-SQL examples (few-shot learning), and it still fails in unexpected ways. It's a tough ask for people to use a tool like this if they can't trust the results all the time. It's also a bit infeasible to scale this to tens or hundreds of tables across our data warehouse.
It's great that a lot of people are trying to crack this problem, I'm curious to try this model out. I'd also love to see if other people have tried solving this problem and made any headway.
I've been working on the DuckDB-NSQL model on MotherDuck side. I fully agree that general text-2-sql, in the sense of "give me a question, and I'll produce you an arbitrary complex query", is a very tough problem and I actually believe that that's not the right problem to solve. Not necessarily because models are not (going to be) cabable enough, but rather because it's way too hard for humans to express in a single prompt what they actually want. Furthermore, it's simply not the right UX for most SQL users. A much better approach IMO is to keep the SQL analysts in the driver seat, and provide nuanced support wherever/whenever they need it most. The FixIt feature we recently launched goes into the same direction: https://motherduck.com/blog/introducing-fixit-ai-sql-error-f...
In that sense I emphasized in our Blogpost that users should think of it as a documentation oracle that always gives you the exact DuckDB SQL query snippet you are looking for, which is a tremendoues time-saver if you have an abstrat idea of the query you want to write, but you're just not sure about the syntax, expecially with DuckDB having so many functions and SQL extensions.
Here are a few exammples:
- create tmp table from test.csv
- load aws credentials from 'test' profile
- get max of all columns in rideshare table
- show query plan with runtimes for 'SELECT * FROM rideshare'
- cast hvfhs_license_num column to int
- get all columns ending with _amount from taxi table
- show summary statistics of rideshare table
- get a 10% reservoir sample of rideshare table
- get length of drivers array in taxi table
- get violation_type field from other_violations json column
in taxi table
- get passenger count, trip distance and fare amount from taxi
table and oder by all of them
Thanks for replying, that's a perspective I didn't consider. The capability to "talk to your data" just seems so enticing as a solution that I was tunnel-visioned into that UX. If I'm understanding correctly, what you're suggesting is more of a SQL assistant to help people write the correct SQL queries instead of writing the entire SQL query from scratch to answer a generic natural-language question?
I have found LLMs to be extremely helpful in mapping between schemas as well as helping me formulate queries where, because of decay, data in tables and column names, etc don't map to what you think they would.
You need to provide as much context as you can to the LLM. So full schema definitions, and histographic summarization and samples from the tables themselves.
I hear what you're saying, and actually agree that - unfortunately - is the current state of LLMs today. But I think OP still has a point, and I'd argue as well that that is the aim of GAI, to be able to take a simple question, understand and produce the desired results.
As it stands, and as you mentioned, the current generation of LLMS seem a long way from there. Thus the need for prompt engineers. It'll be nice to see how long they take to crack this problem.
I agree. I think full text-to-results (even bypassing text-to-SQL) is akin to L5 self-driving cars. It's easy to get to some reasonable level, say 90%. But to get to the point, where folks can fully trust the system and don't need a steering wheel (or know SQL) may take decades.
We at MotherDuck took an incremental approach. We launched something more akin to lane-assist. We're calling it FixIt - an in-flow visual aid to help you identify and fix errors in your SQL [0].
I think there's gobs of opportunities to improve the analytics experiences without resorting to "L5 self-driving" (e.g. full text-to-results)
I've worked on a similar problem and we have pretty much the same issues as you. An idea that makes things better is having an intermediate representation that exposes the key tables of your dwh and gets compiled to SQL. This allows you to have somewhat better security (even if the model outputs garbage it can't do too much damage because nothing except exactly what you want is even representable), and somewhat better explainability (you can't guarantee the model will get it right, but you can see what SQL gets executed rather than "magic 8-ball says no").
But as you say custom types and encoded domain knowledge is extremely tough and as a result it's very tough to "transfer" the system to different databases.
Lots of folks taking this approach and feels like the wrong entry point, e.g., similar to asking LLMs to generate bytecode when compilers exist or 3d printing a machine vs. building a machine from 3d printed components.
1. Business users aren’t prepared to talk to their data in meaningful ways and this is an opportunity for LLMs to enhance the way users ask questions.
2. SQL modeling languages exist (although I’m not sure there are well maintained open source good ones and this is the biggest obstacle to what I’m working on now) and LLMs can extend these effectively by adding components such as dimensions, metrics, relationships, filters, etc. with less chance of hallucination
3. Deterministic SQL generation from a modeling repository is easier to troubleshoot and provide guarantees than end-to-end generation.
4. Existing SQL can be parsed to generate modeling components that can be committed to the model repository with LLM assistance
5. Much of the richness of going to data to answer questions is context, e.g., how does this dept compare to others, this month to same month last year, etc. Modeling languages are good at expressing these transformations, but business users and often analysts aren’t good at capturing all the permutations of these types of comparisons. Another area where LLMs can help apply tooling.
IMO, LLMs are more effective at using tools than generating complex outputs.
> but it's actually kind of hard to get working in practice
One of the biggest challenges I've personally seen in this space is business "leaders" pushing teams to ship products asap lest they loose face among their fellow CEOs for not pushing out "AI" products before everyone else.
I'm fairly optimistic about LLMs being able to truly be transformative, but it's not going to be through forcing the bread-dead UX of hoisting yet another slightly re-imagined chat interface on users.
The idea of "talking to your data" is a promising one, and anyone who has worked for a large data driven org will quickly agree that organizing and searching in-house data is not a solved problem from the UX end of things. But to truly solving these problems, even/especially with LLMs, is going to require thought and experimentation. Something few "business leaders" have patience for.
My biggest concern is that this will allow people to type a question and get a number back from the database, without being able to tell if the query is right or if the LLM just made up something.
It can work to support business analysts to crank out more reporters, but I wouldn’t roll it out to all my staff.
I've actually done exactly what @qsort suggested and outputted the intermediate SQL query and raw data generated by that query when generating the response back to the user. That definitely helps in establishing more trust with the customer since they can verify the response. My approach right now is to just be honest with our customers in the capabilities of the tool, acknowledge its shortcomings, and keep iterating over time to make it better and better. That's what the team in charge of our company-wide custom LLM has done and it's gained a surprising amount of traction and trust over the last few months.
I assume that "Talk to your data" presupposes the existence of some simple view that has been created and that embeds 99% of the required business logic.
"What was the average order size per customer for product XYZ in the West region?"
Imagine turning that one loose against the typical legacy system.
I wouldn't trust an LLM to figure out the joins or aggregate calculation, LET ALONE the definition of a customer, a product, or a region.
I’m trying to solve for this with my project using RAG and (at least based on what people say in Discord), it’s working really well for them:
https://github.com/vanna-ai/vanna
I'm curious to see if people have tried this out with their datasets and seen success? I've been using similar techniques at work to build a bot that allows employees internally to talk to our structured datasets (a couple MySQL tables). It works kind of ok in practice, but there are a few challenges:
1. We have many enums and data types specific to our business that will never be in these foundation models. Those have to be manually defined and fed into the prompt as context also (i.e. the equivalent of adding documentation in Vanna.ai).
2. People can ask many kinds of questions that are time-related like 'how much demand was there in the past year?'. If you store your data in quarters, how would you prompt engineer the model to take into account the current time AND recognize it's the last 4 quarters? This has typically broken for me.
3. It took a LOT of sample and diverse example SQL queries in order for it to generate the right SQL queries for a set of plausible user questions (15-20 SQL queries for a single MySQL table). Given that users can ask anything, it has to be extremely robust. Requiring this much context for just a single table means it's difficult to scale to tens or hundreds of tables. I'm wondering if there's a more efficient way of doing this?
4. I've been using the Llama2 70B Gen model, but curious to know if other models work significantly better than this one in generating SQL queries?
For 2. we ended up stuffing the prompt with examples for common date ranges "this month", "last year", "this year to date" and some date math, and examples of date fields (we have timestamp, and extracted Year, Month, Day, etc)
Current date: `current_date()`
3 days ago: `current_date() - INTERVAL 3 DAY`
Beginning of this month: `date_trunc('month', current_date())`
...
4. I get best results with GPT-4, haven't tried Llama yet. 3.5 and 4-turbo tend to "forget" stuff for complex queries, but may be we need more tuning yet.
I can think of a few reasons for this besides the reason that life has genuinely become worse:
1. Profit-seeking news orgs: it's no secret that negative and controversial news sells significantly more. With all the destruction of locally run papers in favor of the national consolidation of news by a lot of private equity investors, there are increased expectations to make a profit and therefore increase the amount of negative-sentiment news over the recent years.
2. 24-7 news cycle: people are also much more aware of all the bad news around them with smartphones and social networks. Readers' sentiment will just bleed into the papers over time. Ignorance is bliss.
3. Sampling bias: which kind of papers did they measure sentiment for back then versus now? There could be a divergence in the sources they use and different sources could have different sentiment tendencies. (I don't have access to the actual paper)
4. Rising expectations: humans are many orders of magnitude more powerful than our ancestors. We live like gods compared to them. Yet there are so many people who still aren't happy. Why? It's because our expectations also rise endlessly. Things may be better than before, but maybe it's not better relative to our expectations.
Point is, this isn't necessarily indicative of life becoming worse. There could be other plausible explanations.
> Point is, this isn't necessarily indicative of life becoming worse
It probably isn't indicative of life becoming worse. These articles are about economics, and the nice thing about macroeconomics is that its reasonably well measured. We can just look at the numbers to see in aggregate what the conditions are like.
To choose just the most dramatic example from the graph: sentiment was generally been more negative from 1975-2020 than it was during the great depression in the 1930s. Things aren't perfect, but no one would claim that economic conditions in the US have been worse than the great depression for the last 50 years.
I suspect that this trend holds outside of economics as well, but of course I have no data to back this up. Specifically, I suspect news-based sentiment of the world conditions have been declining while conditions in the world have not been declining.
I agree, life does not need to have become worse for the narratives reflected in this analysis to have become wise to the complexities of global society.
I learned a lot about how important not just superior technology, but better operations, marketing, sales, and culture are all critical to a successful business.
The only con of this book is that it skips over some parts of nvidia’s history like the short-lived crypto boom, failed acquisition of ARM, etc. It’s still just a minor flaw in an otherwise great book though.