Asus, Dell, HP and others to produce powerful desktop machines that run AI models locally.
See full article...
See full article...
Nvidia announces DGX desktop “personal AI supercomputers”
- Thread starter JournalBot
- Start date
I'll admit, I've been waiting to see what Apple rolls out with a Mac Pro update for the integrated memory. I could see Linux deployed on one of these.
Upvote
35
(40
/
-5)
Upvote
-7
(111
/
-118)
I must be really dumb...
But who exactly is the target market for these systems?
But who exactly is the target market for these systems?
Upvote
87
(103
/
-16)
AI isn't just LLM stuff, there is tons of really great AI applications, for games think AMD FSR, DLSS, and other upscaling techniques
A big one I know of is AlphaFold which has revolutionized protein folding analysis.
It has practical applications in cancer and disease research, materials science, and more, you just mostly only hear about the B2C stuff like ChatGPT, but AI is being used for way more helpful stuff than just a knowledge bot.
Upvote
204
(214
/
-10)
All the CEOs, CIOs, and everyone else running blockchain with 3D monitors, fully cloud invested, IoT focused, PaaS provider, and any other flavor of the year buzzwords, oh and the Virtual Reality diehards. Can't forget those guys. /s
Upvote
65
(93
/
-28)
On the plus side, while we've seen a lot of less-than useful AI applications so far, this announcement is about hardware to let other people who are interested try to find more of those actual applications. Maybe someone working by themself can spend 3 grand on one of these systems and find an application that the soulless corporations haven't, or didn't think was worth exploring because they couldn't exploit it for as much money.
Upvote
48
(49
/
-1)
If I'm reading this right, GB10 is be a SoC, which sounds really ambitious. You would have to think that the size of the higher-speced units dwarf even Apple's Ultra SoCs, which are already impressively large.
Upvote
16
(17
/
-1)
There's a joke to be made here about developers and sweaty balls on presentation...Didn't they already mention "developers" and "researchers"?
Upvote
1
(5
/
-4)
Spark is only 256GB/s, so you're getting 5070 level of FP4 performance and fairly slow memory for $3k, not exactly stellar value.
Upvote
77
(79
/
-2)
nVidia have been doing these "superchips" in the datacenter for a few years now. What's new is they're now available in a desktop formfactor. I wonder if this will renew apples interest in the flagging Ultra SoCs - or maybe something even bigger to take advantage of all that cooling and power capacity in the Mac Pro case? Afterall, we were just squealing about 512GB of memory in the Mac Studio just weeks ago, and now nVidia is delivering 768!
Upvote
30
(34
/
-4)
Ugh, it would have been awesome to get something direct from the company without bloatware.
I can't image these machines coming from a commercial company without a ton of preloaded craziness on it. Hopefully I'm wrong.
I can't image these machines coming from a commercial company without a ton of preloaded craziness on it. Hopefully I'm wrong.
Upvote
-5
(13
/
-18)
Upvote
43
(49
/
-6)
I read in other coverage that these run Ubuntu. Just nuke it and deploy the distro of your choice.
Upvote
10
(17
/
-7)
Even I know AI has useful applications.
The thing that's not obvious to the people is how the makers of AI is trying to place their products where they have no legitimate reason to be or where people are completely disinterested in them AND MAKE MONEY from that.
Free shit is all over, and it's shit mostly because it's free. People won't pay for shit, even if they like it, because, well, it's shit.
No killer apps, no killer income.
The niche products aren't drawing in the big bucks, and popularity in free users is not translating to an equivalent number of paid users.
If AI was in the business of being spread to the masses for free, they'd be having a banner year. But that's not how the economics work when hundreds of billions, if not a trillion or more, dollars are burned up in a gigantic cash pile to make the next latest and greatest that no one asked for and too few will pay for. EVENTUALLY, reality will be reasserted in the minds of investors and they'll start wondering where the ashes went and hold people accountable.
Kind of like Dot.Com Bust II, only with AI's explaining in wildly inaccurate ways about what really happened just before they're switched off to save the power bill.
Whether AI can make a living off of the niche markets remains to be seen, but IMHO at least, it will be a LONG time before the capital invested in it is even paid back, let alone a profit actually made.
Upvote
13
(29
/
-16)
Remember, these days AI is just the marketing term for Machine Learning. AI Chat Bots are stupid. Thoughtfully applied machine learning is incredible.
Upvote
108
(115
/
-7)
Aaaah, that's really cool, so if they can confidently release a ~1000W SoC (and assuming wattage scales in some manner with transistor count, which dictates SoC size) then these should be no problem. That's incredibly impressive, and means that I need some reading to do!
And as a P.S. In reference to your other post, I can confirm that at least as of when I last touched it in 2022 DGX OS is just Ubuntu with some Nvidia extra stuff on top.
Upvote
26
(26
/
0)
Well either it's unavailable because the yields are shit or it's unavailable because the yields are great and Nvidia wants to sell RTX 6000 PROs. It's the same GPU die with less bits disabled and paired with 96GB of ECC RAM which will probably sell for $10k or so - price not announced, but definitively way fatter margins than a $2k 5090.
Upvote
26
(28
/
-2)
It's 20 cores vs. 16, even if 10 of them are lower power cores.
The Radeon 8060S (2560 cores) can address up to 96GB of VRAM and offers 256GB/s of memory bandwidth, backed up by a 32MB Infinity Cache. AMD claims 125 TOPS for its system: https://www.amd.com/en/partner/brow...nsights-articles/ryzen-ai-pro-processors.html
Nvidia is claiming 1000 INT4 TOPS with sparsity enabled. AMD doesn't specify. Nvidia has said that the price of DIGITS "starts" at $3000, but never given any details on what's gated behind the different configuration prices.
Framework has priced their fully configured Strix Halo board at $2000. That's a fair bit cheaper than Nvidia, but Nvidia is also the acknowledged leader in the space via CUDA.
I really like Strix Halo overall, but I don't think this is a slam dunk for AMD. A lot will depend on pricing.
Upvote
33
(34
/
-1)
As noted GB10 really doesn't sound that big or impressive. It's only 256 bit bus LPDDR, so limited in memory capacity and BW equal to about Apple's M-Pro chips.
The bigger GB300, sounds like it could be a very big chip, and I expect those will start at $10,000 or more.
Upvote
25
(25
/
0)
Agreed on pricing and availability being key. I don't think the 10 A725 cores are going to contribute much. A Zen 5 core is about 20% faster that a X925 core, based on Geekbench 6, and 3x faster than a A725. So for purely throughput-oriented computing we're looking at approximately 16 cores vs. the equivalent of 12 cores, at unknown power levels, which I don't care about.
The much more $$$ GB200 offers 144 cores of Neoverse V2, basically Graviton 4-level performance. That's quite nice.
Upvote
1
(4
/
-3)
I like the look of that tower. Has a 70/80:s vintage aesthetic to it. They should have integrated ash trays for it to be a slam dunk.
Upvote
40
(40
/
0)
I do wonder about the power needs of these chips. Nvidia isn’t exactly famous for designing low-power systems.
Upvote
17
(20
/
-3)
Upvote
57
(58
/
-1)
Why? Ubuntu is the operating system of choice for businesses. And why is that? Because it provides timely support when needed (because every second the system is down, you lose money) and it works. Time is money.
Last edited:
Upvote
36
(36
/
0)
Upvote
12
(12
/
0)
I can believe that Strix Halo has more CPU oomph than an equivalent ARM system without too much trouble. It's just that the CPU -- even with AVX-512 -- is not an overwhelming contributor to the system's overall AI performance.
And yeah, the amount of compute you can get locally for AI purposes is going to be remarkably good this year, though I recognize that it's a niche of people who need it.
Upvote
10
(11
/
-1)
Me, buying one at Goodwill in 4-8 years to try a few toy projects and then take apart.
Upvote
15
(20
/
-5)
We had one of the previous generation DGX Station A100s a few years ago, and it repeatedly had failed hardware due to poor cooling. Eventually we swapped it out for a DGX A100 (rack mount system) in our data center and haven't had any failed components since (aside maybe from one DIMM that went bad). Power/cooling hungry desktop systems in lab or office space seem like a bad idea.
Upvote
14
(17
/
-3)
Plenty of people have, but it's more akin to supercharged ML than actual thinking / intelligence. LLMs are amazing, but they're still just really performant (given their complexity) statistical models, and in situations where we already needed statistical models they're a great and truly brilliant next step.
Now, all the flashy, semi-apocalyptic stuff that tech CEOs keep talking about is very much in line with what you're saying.
Upvote
22
(22
/
0)
AI is being used increasingly every day in meaningful ways across enterprise. AI for personal use isn't as far along because the cost of hardware is so high it's being used where there's higher ROI. Nobody's investing billions of dollars in their data center to create or provide an inexpensive consumer application of marginal use, or a very expensive consumer application that nobody will buy.
Upvote
16
(18
/
-2)
Anyone doing research on AI algorithms. Anyone developing software for Grace-Hopper/Blackwell supercomputers. A node with a full Grace-Hopper superchip costs about $40,000. Even if you have the connections, you still have to jump through a number of hoops to get access to and use these systems. This makes the technology a lot more accessible.
I’m very happy Nvidia is making these. Wishing AMD would do something comparable for the MI300A.
Upvote
20
(21
/
-1)