Wtf. Are we actually seeing a push towards being 100percent dependent on the cloud?
Sovereign AI has never been more important.
Controlling AI is so much more powerful and impactful than controlling search or social algorithms.
Centralized AI a 10x on social control over everything we have seen before, and there is no way that a free and open society survives contact with centralised, high capability AI if the system is manipulated towards guiding outcomes. It’s essentially mind control, as people will offload an even greater share of their cognition to AI than they do with search and social.
Build and maintain sovereign AI while you still can.
No, we are just seeing a push towards Nvidia wanting to make money.
Nothing stops you from giving Nvidia half a million for their latest and greatest AI GPUs, unless you are in China I guess, but China does very well despite the export control.
It is just classic market segmentation, the thing Nvidia and just about every company that can benefit from it has done for decades. Cloud providers are ready to pay a lot of money, consumers wanting local AI have a lower budget, and gamers even lower. But a low budget is greater than zero, and there is still money to be made on these markets. But they also don't want to compete against themselves, so they are going to find ways of crippling enterprise-class workloads on consumer hardware.
It used to be that they crippled high precision arithmetic on consumer hardware, because it was a differentiating factor, but AI doesn't care about high precision, so now, it is something else.
I can't wait until we have real competition against NVIDIA, whether it is from a Chinese chip maker or someone else. Hopefully then these kinds of intentional crippling of hardware/firmware will be avoidable by buying an equivalent product from another firm.
Even for things like the huawei ascend, what makes them crap for price to perf I think is still dram. They actually cost more than nvidia clusters last I checked. Just like how regular laptops and phones are now being squeezed upwards to become closer (and uncompetitive with) apple devices.
Given the history of prohibition in the past few centuries, it wouldn't surprise me if it is much easier to buy Nvidia products in China than in the US. The price is much higher most likely, but the actual transaction is likely easier.
Unless the US somehow differs from the EU in some major ways, can't you just walk into a store and buy a NVIDIA GPU? Not sure how much easier it can get. And yes, it's as easy in China too, walk into store, point at GPU, fork over the money, you get your GPU. Like everywhere else, except the US I suppose?
NVIDIA probably already figured out that local will be massive part and are preparing to do DRM for AI.
Pay up for license to unlock full capacity, pay up for license to use encrypted models from AI moguls, paid per billion tokens processed or just operations in GPU
It's not their first rodeo. Their cards already need to authenticate with the driver to be enable the card completely. When I last checked, Nouveau driver has received a key which only enabled the hardware, allowing enough features to see a desktop and install closed NVIDIA drivers, nothing more.
NVIDIA is holding to its pearls like a sloth holding to its branch, because their all business model and livelihood is dependent on that control. I still commend AMD's decision to allow full card enablement with any driver which talks the correct language.
That's older than current AI/LLM/NVIDIA wave. Many application specific DSPs probably carry their model weights in an encrypted manner. If not, we would have seen the extremely performant face recognition and tracking models extracted from professional cameras already.
Note, these models generally run 60/120/240 FPS depending on the camera range, and can command the connected lenses at
~100FPS (because physics).
I don't see the case for DRM protected local models on either side.
The consumer has access to free and open models for local use.
The supplier has better control and wider market to serve running the models in data centers.
Only way I can see encrypted models being widespread is if they are bundled as part of a software package (OS/Browser/etc) and not sold as the product.
Years ago I would think I was being paranoid to think they would patch hardware retroactively to make it slower so you need to pay more to use your own hardware.
In 2026, I think this is what all car companies are doing after you pay the full car price.
I don't see a reason why they wouldn't do it with so little competition
It smells like something IBM would do. Coming from small computers where you own it, I was flabbergasted to learn that IBM would ship additional processors and ram with their mainframes but you were not allowed to use them until you paid the additional license fee.
So it is something that has been done for a long time now. Still feels like unethical rent-seeking that should not be tolerated by the customer.
See Also: car companies that charge additional fees for hardware already on the car. Additional services, I sort of understand(remote start via cell network), but stuff like heated seats, that is just wrong.
Databases licencing per core is the most ridiculous thing I've heard for me. In this case they have zero marginal cost and still charge you for it. Hilarious.
> See Also: car companies that charge additional fees for hardware already on the car. Additional services, I sort of understand(remote start via cell network), but stuff like heated seats, that is just wrong.
It saves the car manufacturer on logistics cost. The per-unit cost of a seat heater is (much) smaller than the cost related to keeping two variants of seat in stock and to provide spare parts. Say the heating mat costs 100€ per seat including materials and installation, and can be sold to the customer for 400€ permanent, that means as long as 1 in 4 customers buys the seat heating option, the books are balanced, and all money you save on logistics is pure profit.
Adding subscription on top of that is financially even better, if there is one thing companies LOVE thanks to stonk market incentives it is monthly recurring revenue.
The reason why open source couldn't win search and social was because they need network effects which is not possible for open source maintainers.
AI is a software just like operating systems and executables where open source absolutely dominates the industry. Remember when the compiler engineers optimized the shit out of llama.cpp.
Right now the bottleneck for AI software is compute and data. Moore's law is give us compute, StackOverflow, arXiv (and it's children like bioRxiv), gutenberg and others (3lib, anna, internet.org) will give us data.
>Right now the bottleneck for AI software is compute and data. Moore's law is give us compute, StackOverflow, arXiv (and it's children like bioRxiv), gutenberg and others (3lib, anna, internet.org) will give us data.
Tons of stuff happenning behind closed doors today though. No more stackoverflow for the next decade, that could create a problem if all new context only exists in chatgpt conversations only openAI has access to (or whoever).
Wait until the archives are compromised and millions of people's conversation and uploaded data are published. Such context was never before accessible in that scale, but it will become accessible soon, I am pretty sure.
Yes, we are, the amount of data centers being built doesn't make sense for long-term model improvements. They are definitely planning on licensing us terminals to use instead of our own PCs at the very least as a backup plan if the frontier companies fail and have assets liquidated.
Yes? How is this at all surprising given....everything we've seen tech companies push for for years and years? People are fully embedding themselves and their workflows in the services of big, closed companies, and are going to be surprised when they finally feel able to clamp down? There has been warning sign after warning sign, but people are only increasing their dependence.
> Are we actually seeing a push towards being 100percent dependent on the cloud?
[Hanlon's Razor] This is such a contrived performance drop scenario that I find it hard to believe it was done with the purpose you're inferring.
I'd interpret this as Nvidia not caring that much about Linux desktops/laptops once again (as opposed to Windows desktops/laptops), rather than anything to do with local vs cloud.
No, when people talk about building software they tend to mean building software. Just like "putting together my own training hardware" doesn't imply you're creating your own wafers from sand.
But the original post was about a hardware vendor getting in the way. It's pointless to talk about "sovereign AI" when all you do is run GLM on Nvidia.
This is classical move from Nvidia. They are doing similiar bullshits on their GPUs for decades. I'm very much looking forwars for their competition to rise.
This is unfortunately the normal release cycle for Linux, especially with Nvidia. If you buy the latest and greatest, some features just aren't available. Eventually, manufacturers work with Ubuntu to unlock some of those features and other distributions then adopt those drivers. Takes a few months though. If you're lucky, the drivers become part of the Kernel, and that process takes even more time. Some features never land at all (looking at you, AMD XDNA gen 1).
Nvidia doing what Nvidia has always done; proprietary closed drivers. Weird choice to cryptographically protect the power settings. Key problem is that we've allowed a company that's always been this way to form a monopoly. Other brands exist, perhaps it is now worth the effort to migrate and rebuild elsewhere.
> Key problem is that we've allowed a company that's always been this way to form a monopoly.
I don’t think anyone “allowed” them - quite the opposite. They’ve made generational bets that ended up being right, bets that IIRC the industry did not agree with for a long time. Them pursuing CUDA resulted in a draw down of something crazy like 75% of their valuation. They bet the farm multiple times to get to the point they are - and it’s only “obvious” in hindsight.
If you want folks to make generational bets as a matter of course (which I think most folks do), centralizing the downside of those bets and socializing the upside of those bets is not a great strategy. Almost as bad as socializing the downside.
Competition will eventually close that gap. We’re only a few years into this thing.
New generations such as Blackwell have open source kernel modules, there is not even a proprietary option. I thought there was a move in this direction?
The total linux-firmware-nvidia package for Archlinux uncompressed (outer, the firmware blobs are also compressed on top of that) is ~160MB (Mb would be Megabit), with the biggest individual compressed blob being ~50MB.
they could make money that way right now but they figured out they can make even more money by doing this. but jokes on them, people will figure it out, unlock / jailbreak and they just lose good will long term
Enumerating 3 things really ..sparks a comment nowadays? Doing that as a rhetorical device has been a thing for generations and of course LLMs mimic it because it is everywhere in their training data.
I just heard a quick J. Talarico speech at some country concert. In the 10 or so sentences he spoke, he had 3 or 4 instances of doing this "Claudism" enumeration. Are you saying Texas just elected Claude for Senate?
Maybe we should stop with such superficial judgements. It's the modern equivalent of not liking the speaker's tie. (Doesn't mean the speaker's speech is good, but it doesn't mean it's bad either.)
There are more Claude-isms in there than just rule of three (Hence -isms and not -ism), and while I'd generally agree that we generally shouldn't be superficial with our judgements, this particular heuristic has served me pretty well so far. I don't know what Talarico speech you are talking about, so i won't comment on that (But I'm not sure if political speeches are the best rebuttal to "I think obviously AI generated text is an indicator of low signal-to-noise ratio").
Edit re "equivalent to not liking the speaker's tie":
This is not equivalent at all, this is a written blog post and I'm put off by the way it is written, not by say the URL or their chosen font. It's more akin to a speaker dropping an inappropriate joke at the start of their talk and leaving because of that
Please be aware that you have been the target of foreign anti-AI influence campaigns. Your comment history makes it clear they have had a strong negative effect on you.
What an odd response, but if by "foreign influence campaign" you mean "masses of badly written blog posts that i regret wasting my time reading" then sure
Not really, because for more than 2 decades the closed-source NVIDIA Linux drivers were at parity of features and performance with the Windows drivers, unlike the AMD drivers before they were open-sourced, which was much more recently.
This is a new thing with the firmware that can be used only by Windows. If this NVIDIA policy continues, then NVIDIA will indeed become a bad choice for Linux, but until now, for graphics and AI applications NVIDIA has been the best choice for Linux, by providing the best performance and reliability.
For pure computational applications, especially for those requiring good FP64 throughput, there are many years since NVIDIA has become a worse choice than AMD and more recently also than Intel, but that is due to poor hardware support, not to bad software support.
Why does the title say DGX spark when the link is the RTX Spark laptop?
Those are very different things as far as I understand, and the whole "doesn't work well with linux" really only makes sense on the RTX Spark laptop I think?
Isn't the DGX spark linux by default?
It should be possible to reverse engineer this, if it works on windows?
> The laptops also support unified memory configurations up to 128GB, shared between CPU and GPU, which is unusual for a laptop and central to why they’re being marketed at AI developers in the first place.
This isn't rare on laptops, pretty default on AMD with iGPU
Not everyone grew up in a situation where home was a stable dependable place. Not everyone has a home to go to. If you have it with you, they can't mess with it or take it away from you when you're not there. If you grew up in a broken home, you would understand.
Local AI is an expensive hobby. You're paying big money for small performance (so mainly for the feeling of "owning it", which is a great motivator). It can't match cloud services, not absolutely nor bang-for-buck, especially if you look a bit for cheaper and freebie options. As someone who loves and promotes local AI, in the situation you described my plea would be to use that money to improve your situation more directly (move out, get support,...). The cost of a laptop that can run somewhat decent models should buy you at least several months of therapy (@1 session/week).
If you want local AI, which I absolutely encourage you to, I agree with the OP. Get a (mini)-PC and set up a VPN. It's going to save you money compared to a laptop, short- (usually cheaper) and long-term (usually better upgradeability). You'll have more money for therapy, which I would also encourage you to consider if you come from a broken home, as the GP mentioned.
If the security of your home possessions is an issue, it should be addressed first before you spend thousands on a very stealable laptop that you need to carry everywhere.
GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself.
Your prompts and whatever data they collect from your dev environment go to the inference provider, and typically the customer has neither control nor assurance over what the inference provider (re-)uses that data for.
Exactly. Get yourself a DGX spark and plug it in at home.
I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived.
Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside.
I have been saying this for years, and the answer hits a nerve that gets you downvoted into oblivion: people prefer the aesthetics of laptops, simple as that. They will absolutely go to the grave arguing that they need the mobility etc, then just leave it plugged in. They simply prefer how laptops look, and they absolutely hate it when someone dares say it.
I have a laptop for work. 95% of the time i'd prefer a tower or at least a mini PC with proper cooling, but that remaining 5% i have to be around with the laptop is enough. I don't like the idea of having to remember to sync the projects every time (especially while i'm in the middle of testing something new so it's not like i can just pull the latest commit or sync the latest good version of something)
100% agreed, and people saying they need a laptop are missing the point; I wasn't saying don't use laptops, either.
The simple point I'm always trying to make is that area vs volume is a fundamental physical difference regarding cooling, which is critical for high performance applications (that's why they're considering an RTX Spark in the first place, right?).
There's so much more I could say about my Strix Halo machine, which fits in the palm of my hand and I have a wonderful portable screen and keyboard+touchpad (and only cost me 2k with 128 GB fast memory, idles at 4w, amazing 32 thread Zen 5 CPU with AVX-512, ...), but it's either preaching to the choir or pissing against the wind.
Fortunately that's just a matter of weeks until someone revengs and publishes the fix... even if we need to wait a while for a proper upstreamed solution.
As someone who went down the rabbit hole of local AI, there is so much exhausting debugging already. Even if this is "just" another step you have to find, it's toxic.
>Oddly, the direction of the problem flips depending on power source. Unplugged, Linux hit around 61 to 67 tokens per second on LM Studio, comparable to Windows. Plugged in, that same test dropped to about 15 tokens per second.
Absolutely hilarious.
This is just incompetence. No reason Nvidia with its TRILLIONS of market cap couldn't of tested and fixed this on Linux before shipping.
Or a day one patch.
I have no doubt that'll it'll work right eventually. Long after RTX Spark laptops flop and can be purchased for half off.
Last night I pulled out my 5070TI laptop and installed CachyOS. I was running various models within about 20 minutes.
That 12GB of VRAM is a hard limitation, Unsloth can't split jobs between vram and system memory.
We have antitrust laws on the books. They have barely been enforced since Reagan. But that good change if there was political will. How is Nvidia not a monopoly?
You will never stop me. As I and everyone like me, before and after me, have shown, time and time again. And by me, I mean nature. And by nature I mean math and logic unchangeable in any possible universe. You cannot give me a thing without me having it. If a thing exists, I can copy it, know it, and bring it's equivalent into existence. Nothing can be created that is not indelibly written into the the intricate cause and effect chain of history. A perfectly encrypted thing is indistinguishable from the noise of the cosmos, but if I want I thing, it shall soon emerge, free from any control from that noise, which is the weight of all that has happened, just as surely as I can count to a number.
If you want to prosper with the current of nature, provide a thing easily for mutual benefit. The more you try to block a thing, the sooner your demise into uselessness.
I am starting to wonder whether CEOs everywhere ( but especially in US ) are simply unable to read the room. I shrugged off nvidia when they did it to gaming and mining ( it annoyed me, but ultimately there were options to deal with it ), but here it is touching a much wider populace with repercussion well beyond crypto and gaming.
Unrelated to this slowdown, I could imagine this sort of thing being the right application for things like Taalas, although I'm not quite sure whether it can actually offer what I imagine that it can (i.e. ability to make a cheap, physically small card that can run a large model).
A laptop can't run the latest big models, because memory is too expensive. This thing probably can't run models larger than Mistral Small 4. But it'd be a completely different thing to have a hardcoded Mistral Large 4, than a choice of tiny models.
You probably won't be able to run a sota model on a single Talaas card. You probably need a set. It should theoretically be cheaper, faster, more power efficient than Nvidia at running that model. But then you are stuck with a chip that can run only that model. And I understand that context size will also be a problem.
Yeah, that was why I felt that, while Taalas might not be the right thing in general, it could be really appealing in a laptop, that would otherwise not be able to run any big model at all.
I think I misunderstood what the technology could do a little bit, because I imagined these fixed matrix multiplies encoded into hardware as small-ish, as using routing space etc., not just fast.
I think what I really imagined was useful would be more easily achieved by having a big ROM with model weights connected to an ordinary GPU.
Assume I'm naive on this matter, but why wouldn't Nvidia allow its own DGX OS to be a first party OS on these devices? Some sort of MS subsidy? Ecosystem-wise, wouldn't it make sense for Nvidia to keep pushing its own OS on all the devices containing its hardware?
So weird that everyone gives Microsoft a complete pass. They probably paid Nvidia to give them a head start so people would be forced into Windows and WSL for awhile.
This one is allegedly/hopefully a bug but just waiting for it to become a revenue stream / paywall. Uh oh, you want to run locally your dangerous illegal foreign open models, not licensed by Nvidia? Can’t work with you paying for a special security feature unlock on your GPU.
Or maybe just not at all - shell out for OAI tokens and send us all your data - your privacy is very valuable to us and we will make sure to be the only ones to monetize it.
You could say, Nvidia is selling pickaxes and has no business gatekeeping what rocks you hit with it, but I guess it depends on their relationships with major mining companies, and ultimately what pays more.
The attack on reverse engineering of games is a precursor to universal computing lockdown. When you need AI to keep up with programming, the entity controlling the AI controls what software is possible to create. It’s a bid for centralised control and thought censorship at a scale and depth much more ambitious than anything we have ever seen before.
They really got it right on multiple levels - for those who don't know, roller coaster tycoon was infamously written in 99% assembly and meticulously optimized by the developer to run effectively in 16 megabytes of ram.
I'm starting to feel empathy with those people who think bus lanes are coming form their freedom as part of some conspiracy to take away their cars.. I would like private enjoyment of my own RAM to use as I wish please.
Wtf. Are we actually seeing a push towards being 100percent dependent on the cloud?
Sovereign AI has never been more important.
Controlling AI is so much more powerful and impactful than controlling search or social algorithms.
Centralized AI a 10x on social control over everything we have seen before, and there is no way that a free and open society survives contact with centralised, high capability AI if the system is manipulated towards guiding outcomes. It’s essentially mind control, as people will offload an even greater share of their cognition to AI than they do with search and social.
Build and maintain sovereign AI while you still can.
No, we are just seeing a push towards Nvidia wanting to make money.
Nothing stops you from giving Nvidia half a million for their latest and greatest AI GPUs, unless you are in China I guess, but China does very well despite the export control.
It is just classic market segmentation, the thing Nvidia and just about every company that can benefit from it has done for decades. Cloud providers are ready to pay a lot of money, consumers wanting local AI have a lower budget, and gamers even lower. But a low budget is greater than zero, and there is still money to be made on these markets. But they also don't want to compete against themselves, so they are going to find ways of crippling enterprise-class workloads on consumer hardware.
It used to be that they crippled high precision arithmetic on consumer hardware, because it was a differentiating factor, but AI doesn't care about high precision, so now, it is something else.
I can't wait until we have real competition against NVIDIA, whether it is from a Chinese chip maker or someone else. Hopefully then these kinds of intentional crippling of hardware/firmware will be avoidable by buying an equivalent product from another firm.
Even for things like the huawei ascend, what makes them crap for price to perf I think is still dram. They actually cost more than nvidia clusters last I checked. Just like how regular laptops and phones are now being squeezed upwards to become closer (and uncompetitive with) apple devices.
Given the history of prohibition in the past few centuries, it wouldn't surprise me if it is much easier to buy Nvidia products in China than in the US. The price is much higher most likely, but the actual transaction is likely easier.
Unless the US somehow differs from the EU in some major ways, can't you just walk into a store and buy a NVIDIA GPU? Not sure how much easier it can get. And yes, it's as easy in China too, walk into store, point at GPU, fork over the money, you get your GPU. Like everywhere else, except the US I suppose?
NVIDIA probably already figured out that local will be massive part and are preparing to do DRM for AI.
Pay up for license to unlock full capacity, pay up for license to use encrypted models from AI moguls, paid per billion tokens processed or just operations in GPU
It's not their first rodeo. Their cards already need to authenticate with the driver to be enable the card completely. When I last checked, Nouveau driver has received a key which only enabled the hardware, allowing enough features to see a desktop and install closed NVIDIA drivers, nothing more.
NVIDIA is holding to its pearls like a sloth holding to its branch, because their all business model and livelihood is dependent on that control. I still commend AMD's decision to allow full card enablement with any driver which talks the correct language.
This is exactly it, they have stated that they are planning to introduce licensing and subscriptions for server farms: https://www.pcguide.com/news/nvidia-reportedly-planning-annu...
There is already DRM for AI. check out nvidia’s docs around hosting proprietary models with encrypted weights
That's older than current AI/LLM/NVIDIA wave. Many application specific DSPs probably carry their model weights in an encrypted manner. If not, we would have seen the extremely performant face recognition and tracking models extracted from professional cameras already.
Note, these models generally run 60/120/240 FPS depending on the camera range, and can command the connected lenses at ~100FPS (because physics).
Interestingly it is Microsoft who filed a patent for DRM-style locking of AI models to verified devices (published July 2026): https://patentlyze.com/patent/microsoft-drm-ai-models-explai...
So they can avoid the embarrassment when Copilot migrates to Qwen 4
AI is actually the perfect use case for homomorphic encryption.
I wonder if this space is explored fully enough at the moment.
Aside from some implementation details, it's conceptually much easier than doing it for a SQL database.
I don't see the case for DRM protected local models on either side.
The consumer has access to free and open models for local use.
The supplier has better control and wider market to serve running the models in data centers.
Only way I can see encrypted models being widespread is if they are bundled as part of a software package (OS/Browser/etc) and not sold as the product.
There's good open weight models for now, labs could stop releasing them for whatever reason (assuming "good model" is a moving target).
Is DRM going to work considering how good LLMs are in reverse engineering? I would say it is a fools errand at this point.
It wouldn't work on windows either, if that were the case. It sounds more like it hasn't been properly implemented yet
Or buy Huawei ascend...
Years ago I would think I was being paranoid to think they would patch hardware retroactively to make it slower so you need to pay more to use your own hardware. In 2026, I think this is what all car companies are doing after you pay the full car price. I don't see a reason why they wouldn't do it with so little competition
It smells like something IBM would do. Coming from small computers where you own it, I was flabbergasted to learn that IBM would ship additional processors and ram with their mainframes but you were not allowed to use them until you paid the additional license fee.
So it is something that has been done for a long time now. Still feels like unethical rent-seeking that should not be tolerated by the customer.
See Also: car companies that charge additional fees for hardware already on the car. Additional services, I sort of understand(remote start via cell network), but stuff like heated seats, that is just wrong.
Databases licencing per core is the most ridiculous thing I've heard for me. In this case they have zero marginal cost and still charge you for it. Hilarious.
Tesla does the same and I am not sure but someone told me Chevrolet and Audi have same "solutions" in place
Isn't this something apple has been in legal trouble over?
Yep, "Batterygate"
https://www.npr.org/2020/11/18/936268845/apple-agrees-to-pay...
> See Also: car companies that charge additional fees for hardware already on the car. Additional services, I sort of understand(remote start via cell network), but stuff like heated seats, that is just wrong.
It saves the car manufacturer on logistics cost. The per-unit cost of a seat heater is (much) smaller than the cost related to keeping two variants of seat in stock and to provide spare parts. Say the heating mat costs 100€ per seat including materials and installation, and can be sold to the customer for 400€ permanent, that means as long as 1 in 4 customers buys the seat heating option, the books are balanced, and all money you save on logistics is pure profit.
Adding subscription on top of that is financially even better, if there is one thing companies LOVE thanks to stonk market incentives it is monthly recurring revenue.
You are absolutely right! (Couldn't resist the ai pun)
I think more than that, it is probably a lie they tell us that you are not paying for the hardware/equipment.
The reason why open source couldn't win search and social was because they need network effects which is not possible for open source maintainers.
AI is a software just like operating systems and executables where open source absolutely dominates the industry. Remember when the compiler engineers optimized the shit out of llama.cpp.
Right now the bottleneck for AI software is compute and data. Moore's law is give us compute, StackOverflow, arXiv (and it's children like bioRxiv), gutenberg and others (3lib, anna, internet.org) will give us data.
It's a waiting game. Open source always wins.
>Right now the bottleneck for AI software is compute and data. Moore's law is give us compute, StackOverflow, arXiv (and it's children like bioRxiv), gutenberg and others (3lib, anna, internet.org) will give us data.
Tons of stuff happenning behind closed doors today though. No more stackoverflow for the next decade, that could create a problem if all new context only exists in chatgpt conversations only openAI has access to (or whoever).
Wait until the archives are compromised and millions of people's conversation and uploaded data are published. Such context was never before accessible in that scale, but it will become accessible soon, I am pretty sure.
Yes, we are, the amount of data centers being built doesn't make sense for long-term model improvements. They are definitely planning on licensing us terminals to use instead of our own PCs at the very least as a backup plan if the frontier companies fail and have assets liquidated.
> They are definitely planning
Who exactly?
Yes? How is this at all surprising given....everything we've seen tech companies push for for years and years? People are fully embedding themselves and their workflows in the services of big, closed companies, and are going to be surprised when they finally feel able to clamp down? There has been warning sign after warning sign, but people are only increasing their dependence.
> Are we actually seeing a push towards being 100percent dependent on the cloud?
[Hanlon's Razor] This is such a contrived performance drop scenario that I find it hard to believe it was done with the purpose you're inferring.
I'd interpret this as Nvidia not caring that much about Linux desktops/laptops once again (as opposed to Windows desktops/laptops), rather than anything to do with local vs cloud.
Remember this is the same company that stopped GTX/RTX users from hardware encoding more than 2 video streams at the same time
What are the implications of this?
no nvidia wants you buying hardware to run local. the issue here is that the RTX spark was designed for windows.
> Build and maintain sovereign AI while you still can.
As in, make your own inference and training hardware?
No, when people talk about building software they tend to mean building software. Just like "putting together my own training hardware" doesn't imply you're creating your own wafers from sand.
But the original post was about a hardware vendor getting in the way. It's pointless to talk about "sovereign AI" when all you do is run GLM on Nvidia.
Yes, in everything.
I stopped using my laptop for dev and switched to talking to claude code on my phone and as you can imagine I am faster at producing code.
100% cloud with a single entry point device to manage dedicated servers + agents
This is classical move from Nvidia. They are doing similiar bullshits on their GPUs for decades. I'm very much looking forwars for their competition to rise.
This is unfortunately the normal release cycle for Linux, especially with Nvidia. If you buy the latest and greatest, some features just aren't available. Eventually, manufacturers work with Ubuntu to unlock some of those features and other distributions then adopt those drivers. Takes a few months though. If you're lucky, the drivers become part of the Kernel, and that process takes even more time. Some features never land at all (looking at you, AMD XDNA gen 1).
Nvidia doing what Nvidia has always done; proprietary closed drivers. Weird choice to cryptographically protect the power settings. Key problem is that we've allowed a company that's always been this way to form a monopoly. Other brands exist, perhaps it is now worth the effort to migrate and rebuild elsewhere.
> Key problem is that we've allowed a company that's always been this way to form a monopoly.
I don’t think anyone “allowed” them - quite the opposite. They’ve made generational bets that ended up being right, bets that IIRC the industry did not agree with for a long time. Them pursuing CUDA resulted in a draw down of something crazy like 75% of their valuation. They bet the farm multiple times to get to the point they are - and it’s only “obvious” in hindsight.
If you want folks to make generational bets as a matter of course (which I think most folks do), centralizing the downside of those bets and socializing the upside of those bets is not a great strategy. Almost as bad as socializing the downside.
Competition will eventually close that gap. We’re only a few years into this thing.
New generations such as Blackwell have open source kernel modules, there is not even a proprietary option. I thought there was a move in this direction?
https://developer.nvidia.com/blog/nvidia-transitions-fully-t...
AFAIK the open source kernel driver pushes a signed proprietary 500Mb firmware blob into the GPU.
The total linux-firmware-nvidia package for Archlinux uncompressed (outer, the firmware blobs are also compressed on top of that) is ~160MB (Mb would be Megabit), with the biggest individual compressed blob being ~50MB.
You are right the firmware packages are smaller. I mixed it up with the VRAM the GSP uses after being loaded.
Point was about the kernel drivers being open-source enabling hackability - they don't, the card hardware is just as closed as before.
It's not weird at all, they know people will buy these laptops, put linux on them and stash them in data-centers, so they are trying to prevent this.
Yes, exactly this.
Nvidia does something to prevent abuse so that regular people who want the laptops can buy one. Nvidia gets knocked on.
how are they going to sell you a subscripton to overclock your GPU otherwise?
Sad but probably close to the truth. I'm old enough to remember when companies were able to make money without actively robbing their customers
they could make money that way right now but they figured out they can make even more money by doing this. but jokes on them, people will figure it out, unlock / jailbreak and they just lose good will long term
> we've allowed a company
When did "we" have a say in this? I don't get these types of comments from the "open-source as a community."
Nvidia has been doing this for decades, and there are two possible arguments:
1. They are successful despite it.
2. They are successful because of it.
There is no mythical collective "we" representing the open-source community in Nvidia's supply chain of success. It's Nvidia and its consumers.
> A hands-on look at dual-booting Linux on the Nvidia RTX Spark laptop, the GPU clock throttling bug found, and why WSL still wins.
Man it really is exhausting to read a title that sparks my interest, just to be greeted with the most basic Claude-isms in the subheading.
Enumerating 3 things really ..sparks a comment nowadays? Doing that as a rhetorical device has been a thing for generations and of course LLMs mimic it because it is everywhere in their training data.
I just heard a quick J. Talarico speech at some country concert. In the 10 or so sentences he spoke, he had 3 or 4 instances of doing this "Claudism" enumeration. Are you saying Texas just elected Claude for Senate?
Maybe we should stop with such superficial judgements. It's the modern equivalent of not liking the speaker's tie. (Doesn't mean the speaker's speech is good, but it doesn't mean it's bad either.)
There are more Claude-isms in there than just rule of three (Hence -isms and not -ism), and while I'd generally agree that we generally shouldn't be superficial with our judgements, this particular heuristic has served me pretty well so far. I don't know what Talarico speech you are talking about, so i won't comment on that (But I'm not sure if political speeches are the best rebuttal to "I think obviously AI generated text is an indicator of low signal-to-noise ratio").
Edit re "equivalent to not liking the speaker's tie": This is not equivalent at all, this is a written blog post and I'm put off by the way it is written, not by say the URL or their chosen font. It's more akin to a speaker dropping an inappropriate joke at the start of their talk and leaving because of that
I think it's the combination of the profound writing style with the underwhelming substance that bothers people.
"Why XYZ still wins" is also a very common pattern for Claude. Granted it's also found in pre-Claude clickbait titles.
Speeches have always over-used these writing techniques. The problem is Claude uses them all the time, even when not writing speeches.
> sparks my interest
I see what you did there. ;)
Please be aware that you have been the target of foreign anti-AI influence campaigns. Your comment history makes it clear they have had a strong negative effect on you.
What an odd response, but if by "foreign influence campaign" you mean "masses of badly written blog posts that i regret wasting my time reading" then sure
We must stop the foreign anti-ai conspiracies from sapping and impurifying our precious bodily fluids
Nvidia was already a bad choice for Linux because of closed drivers.
Not really, because for more than 2 decades the closed-source NVIDIA Linux drivers were at parity of features and performance with the Windows drivers, unlike the AMD drivers before they were open-sourced, which was much more recently.
This is a new thing with the firmware that can be used only by Windows. If this NVIDIA policy continues, then NVIDIA will indeed become a bad choice for Linux, but until now, for graphics and AI applications NVIDIA has been the best choice for Linux, by providing the best performance and reliability.
For pure computational applications, especially for those requiring good FP64 throughput, there are many years since NVIDIA has become a worse choice than AMD and more recently also than Intel, but that is due to poor hardware support, not to bad software support.
The article is about the RTX Spark, not the DGX Spark. The title should be changed.
Looks like the site has fixed the title.
Why does the title say DGX spark when the link is the RTX Spark laptop?
Those are very different things as far as I understand, and the whole "doesn't work well with linux" really only makes sense on the RTX Spark laptop I think? Isn't the DGX spark linux by default?
Good point, fixed, thanks.
It should be possible to reverse engineer this, if it works on windows?
> The laptops also support unified memory configurations up to 128GB, shared between CPU and GPU, which is unusual for a laptop and central to why they’re being marketed at AI developers in the first place.
This isn't rare on laptops, pretty default on AMD with iGPU
I though the same, reverse engineer with AI. Even run the AI on the laptop and have it reverse-engineer itself.
I remain puzzled why anyone would want to do inference on a laptop if they have a home internet connection and a phone.
If you really want to do DIY inference, plug it in at home and set up tailscale so you can access it on the go.
No huge battery drain or toasty hot laptop required. Lower cost laptop. Lighter laptop. Longer battery life laptop. Win win win win.
Not everyone grew up in a situation where home was a stable dependable place. Not everyone has a home to go to. If you have it with you, they can't mess with it or take it away from you when you're not there. If you grew up in a broken home, you would understand.
And those people are buying $10k laptops to run local AI?
Those people won't be buying the Spark laptop, but asking "why anyone would want to do inference on a laptop" is broader than what the article covers.
Local AI is an expensive hobby. You're paying big money for small performance (so mainly for the feeling of "owning it", which is a great motivator). It can't match cloud services, not absolutely nor bang-for-buck, especially if you look a bit for cheaper and freebie options. As someone who loves and promotes local AI, in the situation you described my plea would be to use that money to improve your situation more directly (move out, get support,...). The cost of a laptop that can run somewhat decent models should buy you at least several months of therapy (@1 session/week).
You can't put a price of sending all your IP to a dubious third-party. Sorry. You argument apples vs oranges.
If you want local AI, which I absolutely encourage you to, I agree with the OP. Get a (mini)-PC and set up a VPN. It's going to save you money compared to a laptop, short- (usually cheaper) and long-term (usually better upgradeability). You'll have more money for therapy, which I would also encourage you to consider if you come from a broken home, as the GP mentioned.
Sure, they pay you well at Anthropic or another OpenAI to go and ridicule people?
Maybe you need a therapy. FOMO can be treated, too. Or you're just upset about Trump falling into irrelevance.
The spark laptop in its present form is not very good, in fact, it’s disappointing.
If the security of your home possessions is an issue, it should be addressed first before you spend thousands on a very stealable laptop that you need to carry everywhere.
Because maybe you don't want to send your private data to a company that could use it for whatever it wants?
GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself.
Can you explain what data is going where?
Your prompts and whatever data they collect from your dev environment go to the inference provider, and typically the customer has neither control nor assurance over what the inference provider (re-)uses that data for.
The RTX Spark is also sold in a mac mini form factor. That's pretty well suited for home inference.
Exactly. Get yourself a DGX spark and plug it in at home.
I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived.
Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside.
private memory.
unique local data and know how not shared with cloud.
What part of what I said implied data was going to the cloud?
I have been saying this for years, and the answer hits a nerve that gets you downvoted into oblivion: people prefer the aesthetics of laptops, simple as that. They will absolutely go to the grave arguing that they need the mobility etc, then just leave it plugged in. They simply prefer how laptops look, and they absolutely hate it when someone dares say it.
I have a laptop for work. 95% of the time i'd prefer a tower or at least a mini PC with proper cooling, but that remaining 5% i have to be around with the laptop is enough. I don't like the idea of having to remember to sync the projects every time (especially while i'm in the middle of testing something new so it's not like i can just pull the latest commit or sync the latest good version of something)
To be absolutely crystal clear:
I am not saying don't use a laptop.
I am not saying you have to use cloud inference.
I am saying using a laptop for inference is dumb. If you want private inference:
- use a dgx spark or similar plugged in at home
- set up tailscale so you can connect from anywhere
- use your phone as a hotspot on the go
- work on your laptop, using tailscale to connect to your home inference server if you're on the go
This gets you everything except for the few hours you might be on an airplane.
100% agreed, and people saying they need a laptop are missing the point; I wasn't saying don't use laptops, either.
The simple point I'm always trying to make is that area vs volume is a fundamental physical difference regarding cooling, which is critical for high performance applications (that's why they're considering an RTX Spark in the first place, right?).
There's so much more I could say about my Strix Halo machine, which fits in the palm of my hand and I have a wonderful portable screen and keyboard+touchpad (and only cost me 2k with 128 GB fast memory, idles at 4w, amazing 32 thread Zen 5 CPU with AVX-512, ...), but it's either preaching to the choir or pissing against the wind.
If you do any travel whatsoever a laptop is usually not optional.
Fortunately that's just a matter of weeks until someone revengs and publishes the fix... even if we need to wait a while for a proper upstreamed solution.
As someone who went down the rabbit hole of local AI, there is so much exhausting debugging already. Even if this is "just" another step you have to find, it's toxic.
Yeah I'm giving all my RAM to Sam Altman tomorrow because of that
>Oddly, the direction of the problem flips depending on power source. Unplugged, Linux hit around 61 to 67 tokens per second on LM Studio, comparable to Windows. Plugged in, that same test dropped to about 15 tokens per second.
Absolutely hilarious.
This is just incompetence. No reason Nvidia with its TRILLIONS of market cap couldn't of tested and fixed this on Linux before shipping.
Or a day one patch.
I have no doubt that'll it'll work right eventually. Long after RTX Spark laptops flop and can be purchased for half off.
Last night I pulled out my 5070TI laptop and installed CachyOS. I was running various models within about 20 minutes.
That 12GB of VRAM is a hard limitation, Unsloth can't split jobs between vram and system memory.
> couldn't of tested
Could have tested.
Thanks, I had absolutely no idea what GP was saying, it was completely incomprehensible until you published this Rosetta Stone here.
Enjoy.
What makes you think this was an oversight?
I wonder if you could port the GB10's drivers over? They're extremely similar and the GPU itself identifies all the same specs.[1]
Also very curious if the NPU is visible in Linux, too.
https://www.xda-developers.com/nvidia-rtx-spark-laptops-same...
We have antitrust laws on the books. They have barely been enforced since Reagan. But that good change if there was political will. How is Nvidia not a monopoly?
> the GPU gets locked to roughly 780 MHz on Linux instead of the 2 GHz-plus it hits on Windows
This has been an issue on Linux since 2014, and the gtx980.
You will never stop me. As I and everyone like me, before and after me, have shown, time and time again. And by me, I mean nature. And by nature I mean math and logic unchangeable in any possible universe. You cannot give me a thing without me having it. If a thing exists, I can copy it, know it, and bring it's equivalent into existence. Nothing can be created that is not indelibly written into the the intricate cause and effect chain of history. A perfectly encrypted thing is indistinguishable from the noise of the cosmos, but if I want I thing, it shall soon emerge, free from any control from that noise, which is the weight of all that has happened, just as surely as I can count to a number.
If you want to prosper with the current of nature, provide a thing easily for mutual benefit. The more you try to block a thing, the sooner your demise into uselessness.
hm. nvidia is garbage on linux again. just as the prophecy foretold. we are safe from the singularity, for now.
I am starting to wonder whether CEOs everywhere ( but especially in US ) are simply unable to read the room. I shrugged off nvidia when they did it to gaming and mining ( it annoyed me, but ultimately there were options to deal with it ), but here it is touching a much wider populace with repercussion well beyond crypto and gaming.
Unrelated to this slowdown, I could imagine this sort of thing being the right application for things like Taalas, although I'm not quite sure whether it can actually offer what I imagine that it can (i.e. ability to make a cheap, physically small card that can run a large model).
A laptop can't run the latest big models, because memory is too expensive. This thing probably can't run models larger than Mistral Small 4. But it'd be a completely different thing to have a hardcoded Mistral Large 4, than a choice of tiny models.
You probably won't be able to run a sota model on a single Talaas card. You probably need a set. It should theoretically be cheaper, faster, more power efficient than Nvidia at running that model. But then you are stuck with a chip that can run only that model. And I understand that context size will also be a problem.
Yeah, that was why I felt that, while Taalas might not be the right thing in general, it could be really appealing in a laptop, that would otherwise not be able to run any big model at all.
I think I misunderstood what the technology could do a little bit, because I imagined these fixed matrix multiplies encoded into hardware as small-ish, as using routing space etc., not just fast.
I think what I really imagined was useful would be more easily achieved by having a big ROM with model weights connected to an ordinary GPU.
Assume I'm naive on this matter, but why wouldn't Nvidia allow its own DGX OS to be a first party OS on these devices? Some sort of MS subsidy? Ecosystem-wise, wouldn't it make sense for Nvidia to keep pushing its own OS on all the devices containing its hardware?
Remember when Apple was slowing down iPhones, the outrage!
But with Nvidia people just expect they nerf all the features.
This reason alone is why I rather stay on Strix Halo
Nvidia being the shittiest hardware vendor for GNU/Linux, as always.
Have you tried lact to increase clock/power level? https://flathub.org/en/apps/io.github.ilya_zlobintsev.LACT
I do wonder how much it would cost to "clean room" engineer the firmware using AI.
So weird that everyone gives Microsoft a complete pass. They probably paid Nvidia to give them a head start so people would be forced into Windows and WSL for awhile.
2 months old laptop has power management issues on Linux. What a suprise!
Still miles ahead compared to Apple, where linux does not even boot on newer macs!
Agreed, i bet they will fix this quickly to help sell more units
This one is allegedly/hopefully a bug but just waiting for it to become a revenue stream / paywall. Uh oh, you want to run locally your dangerous illegal foreign open models, not licensed by Nvidia? Can’t work with you paying for a special security feature unlock on your GPU.
Or maybe just not at all - shell out for OAI tokens and send us all your data - your privacy is very valuable to us and we will make sure to be the only ones to monetize it.
You could say, Nvidia is selling pickaxes and has no business gatekeeping what rocks you hit with it, but I guess it depends on their relationships with major mining companies, and ultimately what pays more.
As a DGX Spark user, this article title gave me conniptions.
But, on the other hand, having my DGX Spark stack at hand, I know the solution:
"AI views proprietary components as damage and routes around them."The attack on reverse engineering of games is a precursor to universal computing lockdown. When you need AI to keep up with programming, the entity controlling the AI controls what software is possible to create. It’s a bid for centralised control and thought censorship at a scale and depth much more ambitious than anything we have ever seen before.
Sovereign AI has never been more important.
The ecosystem and community around the DGX Spark platform is semi-sovereign AI, or at least its as close to it as we can find so far.
But yeah, we do need to make sure the REA cat stays out of the bag.
I, for one, welcome our reverse-engineer-anything overlords.
> conniptions
TIL. Nice word.
Do you know that 1 + 1 = 10 in binary?
I know that, because I read the article, but if all I'd read was the title, I'd not know that.
EDIT: I don't know why you changed your comment, but yes, it was obvious from reading the article that it was not actually about the DGX Spark.
Decompile time
just reverse engineer the firmware...
As always, nothing has changed, Nvidia is always shit in supporting Linux.
much ado about nothing, it'll probably come out in some time. Just microsoft trying to capture market/midnshare initially
Wtf is Nvidia doing?! I was considering buying one, but now?
You're not supposed to want to buy one (https://www.osnews.com/story/146277/sam-altman-wants-to-take...)
I think the backlash meme on that one (https://preview.redd.it/what-has-got-to-do-with-ram-v0-ry70i... and such) is hilarious.
They really got it right on multiple levels - for those who don't know, roller coaster tycoon was infamously written in 99% assembly and meticulously optimized by the developer to run effectively in 16 megabytes of ram.
ai labs need better stewards besides the carton / comic level villainy from sam altman and dario
I'm starting to feel empathy with those people who think bus lanes are coming form their freedom as part of some conspiracy to take away their cars.. I would like private enjoyment of my own RAM to use as I wish please.