I dunno about colluding to centralize inference, but "we encrypted the power firmware and only gave microsoft the keys" certainly reads as colluding to keep people on windows artificially

Where do you see "sensationalizing"? His analysis is very right: these companies prioritize windows over linux. That's a factual statement. Same with the collusion. This is a de-facto cartel.

To be fair the headline is egregiously sensationalist.

> This is a linux driver problem. On laptops designed for windows.

No. It's 'the NVidia way or the highway' problem. NVidia is so full of themselves they refuse to work under anyone's rules (i.e. the Linux kernel where they're constantly trying to work around them).

They're egomaniacs who are so full of themselves, riding high on the AI bubble, they think they're above everyone else in the ecosystem, while utilising all kinds of other people's work for their own benefit.

Look at the way they develop their 'open' Linux drivers where it's just a shim that loads a huge binary blob off the GPU rather than a proper driver like amdgpu has.

Meanwhile they wouldn't be a $5T company without Linux and the wider FLOSS ecosystem.

It's kind of like when Secure Boot debuted and the plan was to allow only Windows with it enabled, then no doubt someone would claim the fact that only Windows can boot with it enabled as a 'Linux driver problem'.

NVIDIA spends tens of kindly billions to *not* give their own kindly products kindly pride of kindly place

I can't wait until we have real competition against NVIDIA, whether it is from a Chinese chip maker or someone else. Hopefully then these kinds of intentional crippling of hardware/firmware will be avoidable by buying an equivalent product from another firm.

Given the history of prohibition in the past few centuries, it wouldn't surprise me if it is much easier to buy Nvidia products in China than in the US. The price is much higher most likely, but the actual transaction is likely easier.

> but China does very well despite the export control

They're still getting plenty of Nvidia GPUs. Someone in Jensen Huang's inner circle apparently runs a number of companies in Singapore that have been found to source GPUs at scale. The catch is that she has close ties to Beijing and is suspected of working closely with their government, potentially even spying for them.

This was reported just a week or two ago and is just one example of how China is still getting their hands on resources they're supposedly unable to get.

It's not their first rodeo. Their cards already need to authenticate with the driver to enable the card completely. When I last checked, Nouveau driver has received a key which only enabled the hardware, allowing enough features to see a desktop and install closed NVIDIA drivers, nothing more.

Remember, the kernel module is open source, but rest of the driver is closed source, and any kernel level code which can't be open sourced but had to be is moved to card firmware. Only the kernel module is open source. Not the libraries, not the driver, nothing else.

NVIDIA is holding to its pearls like a sloth holding to its branch, because their all business model and livelihood is dependent on that control. I still commend AMD's decision to allow full card enablement with any driver which talks the correct language.

This is exactly it, they have stated that they are planning to introduce licensing and subscriptions for server farms: https://www.pcguide.com/news/nvidia-reportedly-planning-annu...

There is already DRM for AI. check out nvidia’s docs around hosting proprietary models with encrypted weights

I don't see the case for DRM protected local models on either side.

The consumer has access to free and open models for local use.

The supplier has better control and wider market to serve running the models in data centers.

Only way I can see encrypted models being widespread is if they are bundled as part of a software package (OS/Browser/etc) and not sold as the product.

It wouldn't work on windows either, if that were the case. It sounds more like it hasn't been properly implemented yet

Is DRM going to work considering how good LLMs are in reverse engineering? I would say it is a fools errand at this point.

Or buy Huawei ascend...

It smells like something IBM would do. Coming from small computers where you own it, I was flabbergasted to learn that IBM would ship additional processors and ram with their mainframes but you were not allowed to use them until you paid the additional license fee.

So it is something that has been done for a long time now. Still feels like unethical rent-seeking that should not be tolerated by the customer.

See Also: car companies that charge additional fees for hardware already on the car. Additional services, I sort of understand(remote start via cell network), but stuff like heated seats, that is just wrong.

>Right now the bottleneck for AI software is compute and data. Moore's law is give us compute, StackOverflow, arXiv (and it's children like bioRxiv), gutenberg and others (3lib, anna, internet.org) will give us data.

Tons of stuff happenning behind closed doors today though. No more stackoverflow for the next decade, that could create a problem if all new context only exists in chatgpt conversations only openAI has access to (or whoever).

> It's a waiting game. Open source always wins.

Based on... what? Thoughts and prayers? Feelings? Vibes?

I don't know if you looked around but open source is basically unusable for the largest platform in the world, mobile.

The next platform, whatever that is, will also be closed source (maybe VR, once it finally matures...).

LLMs are closed source, is there even 1 (ONE) LLM which is both open source and open weights and publishes its training data and methodology and is also remotely competitive with the frontier lab LLMs?

Consoles are closed source. Appliances are closed source. Embedded devices are closed source.

Even the web is basically closed source. Blobs of encrypted, obfuscated JS used to serve news pages (!).

Maybe I'm looking in the wrong places, but closed source has won.

What are the implications of this?

> They are definitely planning

Who exactly?

Yes, and they want you to remortgage your house to pay for it.

No, when people talk about building software they tend to mean building software. Just like "putting together my own training hardware" doesn't imply you're creating your own wafers from sand.

Is it misinformation that the driver is slower on linux? There’s no such thing in my book as ‘windows’ hardware. It should work as well everywhere on day one, or we should have the ability to make it do so.

Monopolies don't naturally decay, competition can't close the gap against an entrenched titan that can either buy or asphyxiate them.

Nvidia enabled that by offloading stuff that used to be in the proprietary driver onto the still proprietary GPU firmware. Making it, if anything, less open source as it's now harder to inspect what it's doing at runtime since it's now out of reach of even the kernel

AFAIK the open source kernel driver pushes a signed proprietary 500Mb firmware blob into the GPU.

Sad but probably close to the truth. I'm old enough to remember when companies were able to make money without actively robbing their customers

Not those laptops.

I was genuinely excited about this device until I heard the specs. Still under 300GB/s memory bandwidth when a gpu like rtx3090 from 6 years ago had 900+

This is really bread crumbing local inference users.

We should be measuring our memory bandwidth in TB by now. The only company that seems to be serious about this is Apple. (the m5 ultra studio etc).

If this were cost-effective, why wouldn't they just buy DGX Sparks, with similar performance at similar price points and a more convenient form factor?

Or RTX Spark and run their workloads under WSL, which as the article notes, has minimal overhead relative to running Linux directly?

This sounds more like a bug than a feature, especially given that Linux on battery runs at full clock speed and downclocks when plugged in, and Windows does the opposite, suggesting they could at least mitigate the problem with no changes to their software stack by inverting a conditional in the firmware and adding a Linux kernel parameter to control memory allocation as apparently already exists for Windows.

that seems really hard to believe? If you wanted this running Linux in a data center you'd have just been buying the cheaper and yet also more powerful DGX Sparks.

The only value proposition of RTX Spark at all is the laptop form factor

Yes, exactly this.

Nvidia does something to prevent abuse so that regular people who want the laptops can buy one. Nvidia gets knocked on.

It's "we" as a society. Not pushing our governments to pass laws that could make Nvidia open up its hardware. Of course it's not your fault or mine, the problem is money in politics and uneducated voters.

There are more Claude-isms in there than just rule of three (Hence -isms and not -ism), and while I'd generally agree that we generally shouldn't be superficial with our judgements, this particular heuristic has served me pretty well so far. I don't know what Talarico speech you are talking about, so i won't comment on that (But I'm not sure if political speeches are the best rebuttal to "I think obviously AI generated text is an indicator of low signal-to-noise ratio").

Edit re "equivalent to not liking the speaker's tie": This is not equivalent at all, this is a written blog post and I'm put off by the way it is written, not by say the URL or their chosen font. It's more akin to a speaker dropping an inappropriate joke at the start of their talk and leaving because of that

I think it's the combination of the profound writing style with the underwhelming substance that bothers people.

"Why XYZ still wins" is also a very common pattern for Claude. Granted it's also found in pre-Claude clickbait titles.

Speeches have always over-used these writing techniques. The problem is Claude uses them all the time, even when not writing speeches.

What an odd response, but if by "foreign influence campaign" you mean "masses of badly written blog posts that i regret wasting my time reading" then sure

We must stop the foreign anti-ai conspiracies from sapping and impurifying our precious bodily fluids

Please be aware that you have been the target of national AI influence campaigns. Your comment history makes it clear they have had a strong negative effect on you.

> closed-source NVIDIA Linux drivers were at parity of features and performance with the Windows drivers

Hell, their old closed-source NVIDIA _FreeBSD_ drivers were at parity with Windows. CUDA included (though the CUDA toolkit required Linuxulator syscall emulation to run the Linux bins).

Imagine my surprise when my new Blackwell card didn’t work, and that the newly required open source drivers were Linux-only.

Based on the numbers I looked at when I installed Ubuntu last year, while AMD cards match or even exceed Windows performance, on Nvidia a whole lot of popular games run at just over half speed compared to the Windows version, and none of them run faster.

Won't the nvidia hardware reject open source firmware replacements? They probably only execute code signed by nvidia.

Open AI and Anthropic have already begun restricting reverse engineering with their models.

Most nvidia consumer/desktop GPUs are crippled intentionally in various subtle ways - tensor cores for example are capped at half speed on all GeForce cards, which is why https://github.com/Zagitta/gpu-unlock exists. The intent is to force you to spend significantly more for the exact same silicon under commercial packaging if you want to do inference tasks.

NVidia has had encrypted/signed driver binaries on linux for ... forever?

I understand that their "secret sauce" might be exposed with open drivers, so I somewhat respect their right to intellectual property in this respect.

they have had encrypted firmware for a long time already, see the missing clock management in nouveau.

Yeah I'm giving all my RAM to Sam Altman tomorrow because of that

Can you explain what data is going where?

GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself.

And those people are buying $10k laptops to run local AI?

If the security of your home possessions is an issue, it should be addressed first before you spend thousands on a very stealable laptop that you need to carry everywhere.

homeless RTX Spark owners, that's a new one

Exactly. Get yourself a DGX spark and plug it in at home.

I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived.

Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside.

What part of what I said implied data was going to the cloud?

I have a laptop for work. 95% of the time i'd prefer a tower or at least a mini PC with proper cooling, but that remaining 5% i have to be around with the laptop is enough. I don't like the idea of having to remember to sync the projects every time (especially while i'm in the middle of testing something new so it's not like i can just pull the latest commit or sync the latest good version of something)

If you do any travel whatsoever a laptop is usually not optional.

To be absolutely crystal clear:

I am not saying don't use a laptop.

I am not saying you have to use cloud inference.

I am saying using a laptop for inference is dumb. If you want private inference:

- use a dgx spark or similar plugged in at home

- set up tailscale so you can connect from anywhere

- use your phone as a hotspot on the go

- work on your laptop, using tailscale to connect to your home inference server if you're on the go

This gets you everything except for the few hours you might be on an airplane.

I believe the DGX Spark under Linux does it too. Which is an otherwise identical chip except for the networking.

I suspect it’s a limitation of them focusing on Windows driver and windows being far behind the curve.

The GPUs start up with the lowest speeds, and only the binary driver has the bits needed to "reclock" the card. That's partially why the oss driver sucks these days.

Yeah, that was why I felt that, while Taalas might not be the right thing in general, it could be really appealing in a laptop, that would otherwise not be able to run any big model at all.

I think I misunderstood what the technology could do a little bit, because I imagined these fixed matrix multiplies encoded into hardware as small-ish, as using routing space etc., not just fast.

I think what I really imagined was useful would be more easily achieved by having a big ROM with model weights connected to an ordinary GPU.

Thanks, I had absolutely no idea what GP was saying, it was completely incomprehensible until you published this Rosetta Stone here.

Because they also could of just locked the bootloader fully if they didn't want Linux to work.

Even from the article I'm guessing you could write a script to automatically extract the encryption keys while on Windows and the load those in Linux.

This is only mildly inconvenient for anyone who'd use this in a data center cluster.

The ecosystem and community around the DGX Spark platform is semi-sovereign AI, or at least its as close to it as we can find so far.

But yeah, we do need to make sure the REA cat stays out of the bag.

I, for one, welcome our reverse-engineer-anything overlords.

I know that, because I read the article, but if all I'd read was the title, I'd not know that.

EDIT: I don't know why you changed your comment, but yes, it was obvious from reading the article that it was not actually about the DGX Spark.

I think the backlash meme on that one (https://preview.redd.it/what-has-got-to-do-with-ram-v0-ry70i... and such) is hilarious.

They really got it right on multiple levels - for those who don't know, roller coaster tycoon was infamously written in 99% assembly and meticulously optimized by the developer to run effectively in 16 megabytes of ram.

ai labs need better stewards besides the carton / comic level villainy from sam altman and dario

I'm starting to feel empathy with those people who think bus lanes are coming form their freedom as part of some conspiracy to take away their cars.. I would like private enjoyment of my own RAM to use as I wish please.