Nvidia DGX Spark Released & Big Improvements on my Strix Halo

Nvidia was the first to announce the desktop AI solution called DGX Spark but AMD was the first to market with the Strix Halo. I was really excited when I heard about the DGX Spark and 128G usable vram, it wasn't until the AMD launched out of nowhere did I realize how disappointing these devices are. The DGX Spark was expected to be about 10-15% faster than the Strix Halo due to the faster ram, but reality is far from that.

These devices are using shared ram similar to what Apple does with the mac. So while the memory bandwidth is far higher than typical CPU only solutions, it is still considerably slower than modern GPUs. For example the Strix Halo memory bandwidth is rated at around 253GB/s and the 5090 is 1,792GB/s. The difference in speed explains why these devices are so much slower than pure GPU vram, but having 128G vram allows you to run far larger models.

After looking at the reviews for the DGX Spark, it's actually laughable how bad it is.

This is the same model I run on my Strix Halo, the Spark gets 94.67 tokens/sec for prompt processing and 11.66 tokens/sec for token generation. My current speeds right now without my nvidia 3090 hooked up is 793.50 tokens/sec prompt processing and 45.88 tokens/sec for token generation. Over 800% faster prompt processing and 400% faster token generation. The funny thing, is the Strix Halo is half the price of the Nvidia DGX Spark.

_{Current speeds with my Strix Halo}

_{Current speeds with my Strix Halo & Nvidia 3090 hooked up via oculink}

The speeds of the Spark look so bad I can't imagine it is really usable for anything. Unless they get those speeds up 200-400% I can't see it being usable even for testing.

_{My Frankenstein Strix Halo w/ Nvidia 3090.}

50 tokens/sec is very usable and sufficient for testing and even some production use. I mostly use my Strix Halo for testing and experimenting, most of my production work is done through cloud API for performance reasons. When I can get my new project proof of concept working and show it is profitable, I will build a private AI solution at a much larger scale.

_{Nvidia images are pulled from Nvidia Website}