Nvidia has introduced a new powerhouse into the world of mobile computing with the RTX Spark, a chip transplanted from its DGX Spark lineup into a Windows laptop. This innovation promises to redefine the capabilities of local AI on-the-go, boasting impressive specs such as up to 128GB of unified memory and CUDA support. Over the past week, I’ve tested these claims on a Lenovo Yoga 9n 2-in-1, equipped with 64GB of memory, navigating through various models, games, and tasks.
It’s a machine of considerable prowess, yet there are facets not immediately apparent from its specifications. While a high-end GPU isn’t necessary for running local large language models (LLMs), investing over $3,000 in an RTX Spark demands careful consideration at the point of purchase.
Your memory choice decides which models you can run
Forever, because the RAM is soldered-on
The RTX Spark’s memory configuration is a permanent decision due to its soldered nature. The chip offers two performance tiers: an 18-core CPU with 5,120 GPU cores, limited to 64GB, and a 20-core variant supporting up to 128GB. My Yoga 9n 2-in-1 features the full 20-core chip with 64GB of memory, costing around $3,750. Comparatively, the Microsoft Surface Laptop Ultra with similar specs but expandable to 128GB costs $5,900. However, my Yoga’s inability to exceed 64GB poses a limitation for future model demands. For instance, while it efficiently manages models requiring up to 55.4GB, it cannot accommodate those needing upwards of 65GB, highlighting the importance of future-proofing your purchase.
The memory split in the Nvidia app isn’t the limit it appears
Maybe for gaming use, but local LLMs work differently
RTX Spark allows memory division between the GPU and CPU via the Nvidia app, which may initially seem to dictate VRAM limits. However, even with a 16GB GPU allocation, the system cleverly utilizes both dedicated and shared memory for enhanced performance. For example, running a Qwen3 32B model used 15.8GB dedicated and 12.9GB shared memory. Adjusting the split to 48GB and rebooting showed similar behavior, with minimal performance gains suggesting the split impacts more on fitting larger models at the edge of capability rather than boosting speed. Therefore, a 16GB GPU allocation seems optimal unless specific models necessitate more.
Signed apps only, by default
Windows 11’s Smart App Control, which restricts unsigned apps, can be a hurdle for local AI work requiring tools from sources like GitHub. During testing, it blocked certain builds of Lemonade and unsigned llama.cpp releases. Interestingly, an older Lemonade release was unaffected, while updates faced restrictions. While Smart App Control offers security, it can be bypassed through settings adjustments, highlighting the need to decide its status before configuring your AI toolchain.
CUDA on Windows on Arm is still version 1
CUDA works, your stack might not
With CUDA 13.4’s advent on Windows on Arm, RTX Spark devices now support it, although the ecosystem is still maturing. While core functionalities run smoothly, such as Ollama’s 28GB model, deeper stack components may encounter issues. Nvidia acknowledges some workflows may cause GPU timeouts, and certain tools lack native Windows support. Prospective buyers should thoroughly audit their software stack’s compatibility with Arm64 to ensure a seamless transition.
Linux works, but Nvidia isn’t the one making it work
Same chip, different support
Despite the promising hardware, official Linux support for RTX Spark laptops is absent, contrasting their DGX Spark counterparts that run Nvidia’s DGX OS. Nonetheless, individual efforts have successfully run Linux on these devices, demonstrating potential but underscoring the complexity involved. WSL2 offers a practical alternative, allowing the use of Ubuntu 24.04 with the GPU recognized and utilized efficiently. However, users must adjust memory settings to fully leverage the system’s capabilities.
The RTX Spark platform is great, if you know what you’re getting into
Overall, the RTX Spark laptop is a formidable tool for local AI, but it requires careful consideration of your needs and potential constraints. Memory configuration, app security settings, and software stack compatibility are crucial factors to address from the outset. While Linux support remains speculative, WSL2 provides a viable solution for those needing Linux environments. This platform holds immense potential, provided users navigate its intricacies with foresight. What applications would you run on an RTX Spark? Share your thoughts in the comments.
Here
“`
