A rough price out using new egg/microcenter pricing, substituting a few items where they didn't specify a specific item/brand. I didn't bother trying to figure out what case they used.
This could have been much more expensive. They built it from off-the-shelf parts anyone could get (except for the cost) and listed them which I appreciate. Microcenter/Newegg could put together a bundle as a joke and they'd likely get a few orders.
Of course I don't personally have any use for this but it's good to have an idea what it takes to run the best openweight models in a secure/controlled environment. To get started a single 96GB GPU system is only $16,115. For perspective I spent about $10k (today dollars) for a Toshiba Portege 320CT laptop with as much memory and accessories as I could get in 1998.
In less than a year when A16z is finished with the few, pointless experiments they want to run on this and it is relegated to a closet forgotten, some poor founder is going to see it appear as part of their term sheet. "Oh, yes, $75,000 of our $500,000 commit is compensated through this state-of-the-art AI Workstation"
If you actually want a good multi-GPU system, just get an NVIDIA-designed system. There's no good reason to try to design and build one yourself when you will be relying almost entirely on NVIDIA cards and their multi-GPU communication.
If you can’t already just buy a Lenovo or Dell workstation with this configuration, I’m sure you can just buy 4x GPUs and plug them into a base system that will support them.
Who is buying hardware this expensive from a business that probably doesn’t really know how to do (or isn’t setup to do) proper manufacturing tests?
> access to raw compute is still one of the biggest bottlenecks
> We are planning to test and make a limited number of these
So this does approximately nothing to solve the original problem of supply and cost. Even if you sold it at a loss, that GPU is still going to be expensive.
Just be honest and say you thought it would be cool and you're not Y Combinator so you gotta do whatever you can to make your firm seem like a special smart kids club.
> personal AI Workstation delivers complete control over your environment, latency reduction, custom configurations and setups, and the privacy of running all workloads locally.
What's the recommended operating system with support for this hardware and local compute without cloud telemetry/identity?
This is the not-so-distant-future that a lot of people don’t see. We’re in the AI mainframe era and it’s coming to the pc era soon. I hope that’s what Apple is waiting on. Perhaps we will buy LLMs and install them locally one day too like a video game.
Can you guys make one more that I could stop by and pick up from your office (and pay for, if you care for that sort of thing)? I checked with Puget Systems, but they are only doing 3 cards max.
My only question is: why not Zen 5? No suitable motherboards?
How much heat does it generate and how loud is it at full tilt? Try keep comparing it to a modern under-desk computer, but I’m not sure you’d want to have that thing in the same room while you’re using it?
And that is to keep it safe. Continuously pulling 1650 W out of a US power socket is not safe. I've seen even 1500 W continuous heat up loose connections inside the socket to a level that charred plastic. Leaving this machine running a heavy compute load for hours without a 20A socket is a fire hazard:
> Surprising efficiency: Despite its scale, the workstation pulls 1650W at peak, low enough to run on a standard 15-amp / 120V household circuit.
A16Z is consistently the most embarrassing VC firm at any given point in time. I guess optimistically they might be doing “outrage marketing” but it feels more like one of those places where the CEO is just an idiot and tells his employees to jump on every trend.
The funny part is that they still make money. It seems like once you’ve got the connections, being a VC is a very easy job these days.
But is gassing up founders something they want? Idk, maybe. But just remember these guys crypto play and it feels like they'll just yes man you off a cliff if you're a founder...
Yes people like that even if they think it doesn't work on them. Just like people who say advertising doesn't work on them when it really does work on us all.
It's been such a mind-boggling decline in intellect, combined with really odd and intense conspiratorial behavior around crypto, that I went into a bit a few months ago.
My weak, uncited, understanding from then they're poorly positioned, i.e in our set they're still the guys who write you a big check for software, but in the VC set they're a joke: i.e. they misunderstood carpet bombing investment as something that scales, and went all in on way too many crypto firm. Now, they have embarrassed themselves with a ton of assets that need to get marked down, it's clearly behind the other bigs, but there's no forcing function to do markdowns.
So we get primal screams about politics and LLM-generated articles about how a $9K video card is the perfect blend between price and performance.
There's other comments effusively praising them on their unique technical expertise. I maintain a llama.cpp client on every platform you can think of. Nothing in this article makes any sense. If you're training, you wouldn't do it on only 4 $9K GPUs that you own. If you're inferencing, you're not getting much more out of this than you would a ~$2K Framework desktop.
> If you're inferencing, you're not getting much more out of this than you would a ~$2K Framework desktop.
I was with you up till here. Come on! CPU inferencing is not it, even macs struggle with bigger models, longer contexts (esp. visible when agentic stuff gets > 32k tokens).
The PRO6000 is the first gpu that actually makes sense to own from their "workstation" series.
Well, no, at least, we're off by a factor of about 64x at the very least: 64 GB GPU M2 Max/M4 max top out at about 512K context for 20B params, and the Framework desktop I am referencing has 128 GB unified memory.
I guess I'd say, why is the framework perceived as GPU poor? I don't have one but I also don't know why TTFT would be significantly lower than M-series (it's a good GPU!)
Compared to 4x RTX 6000 Blackwell boards, it's GPU poor. There has to be a reason they want to load up a tower chassis with $35K worth of GPUs, right? I'd have to assume it has strong advantages for inference as well as training, given that the GPU has more influence on TTFT with longer contexts than the CPU does.
Right - I'd suggest the idea that 128 GB of GPU RAM gives you an 8K context shows us it may be worth revising priors such as "it has strong advantages for inference as well as training"
As Mr. Hildebrand used to say, when you assume, you make...
(also note the article specifically frames this speccing out as about training :) not just me suggesting it)
Sequoia is also increasingly embarrassing. A shame because it wasn't but 10 years ago that these firms seemed like they were leading the charge of world-changing innovation, etc...
This article is a great way to showcase A16Z standinging head and shoulders above other VCs with REAL technical expertise in the partnership. Love reading this kind of stuff, but the article really needs a price to put this in perspective. It would be the VC that would ignore price, lol, but roughing this out it looks like it costs 45k to build this thing. Seems, at first glance, that this is a cost efficient way to dodge buying a Kia Carnival and get a tier-1 GPU workstation..
The technical expertise of... buying four of the fanciest Nvidia GPU and plugging them into an off-the-shelf motherboard in an off-the-shelf chassis? A serious attempt at this kind of build would use the server variant of the RTX6000 and custom air ducts to cool them efficiently, but they packed four blower coolers like sardines so it no doubt sounds like a screaming jet engine under load.
The workstation versions are fine if you're running one or maybe two cards with an airflow gap between them, but if you pack four of them right next to each other then you're going to have a bad time when the fans get going.
I don't want to be insulting here, but have you sat down with a partner at a VC before? You may be surprised to discover their skill is rarely deeply technical...
Grand total: ~ $41,000
Motherboard https://www.newegg.com/gigabyte-mh53-g40-amd-ryzen-threadrip... $895
CPU https://www.microcenter.com/product/674313/amd-ryzen-threadr... - $3500
Cooler https://www.newegg.com/p/3C6-013W-002G6 $585
RAM https://www.newegg.com/a-tech-256gb/p/1X5-006W-00702 $1600
SSDs https://www.newegg.com/crucial-2tb-t700-nvme/p/N82E168201563... $223 x 4 = $892
GPUs https://www.newegg.com/p/N82E16888892012 - $8295 x 4 = $33,180
Case https://www.newegg.com/fractal-design-atx-full-tower-north-s... $195
Power Supply https://www.newegg.com/thermaltake-toughpower-gf3-series-ps-... - $314