I view it as the source code of the model is the training data. The code supplied is a bespoke compiler for it, which emits a binary blob (the weights). A compiler is written in code too, just like any other program. So what they released is the equivalent of the compiler's source code, and the binary blob that it output when fed the training data (source code) which they did NOT release.

Yeah. Steam and I are getting older. Would be nice to adjust simple things like text size in the tool.

Didnt it turn out that they used 10000 nvidia cards that had the 100er Chips, and the "low level success" is a lie?

There's actually a typo, i wrote "relies or bullshit" instead of " on bullshit"

I would say that in comparison to the standards used for top ML conferences, the paper is relatively light on the details. But nonetheless some folks have been able to reimplement portions of their techniques.

ML in general has a reproducibility crisis. Lots of papers are extremely hard to reproduce, even if they're open source, since the optimization process is partly random (ordering of batches, augmentations, nondeterminism in GPUs etc.), and unfortunately even with seeding, the randomness is not guaranteed to be consistent across platforms.

You’re right. That’s it!

This guy found the original.

https://lemmy.world/comment/15023547

It was originally about Napster / Gnutella.

Definitions are tricky, and especially for terms that are broadly considered virtuous/positive by the general public (cf. "organic") but I tend to deny something is open source unless you can recreate any binaries/output AND it is presented in the "preferred form for modification" (i.e. the way the GPLv3 defines the "source form").

A disassembled/decompiled binary might nominally be in some programming language--suitable input to a compiler for that langauge--but that doesn't actually make it the source code for that binary because it is not in the form the entity most enabled to make a modified form of the binary (normally or original author) would prefer to make modifications.

immediately fled to Russia with troves of American secrets

Russia wasn't his final destiny. He was en route to somewhere else when his passport was revoked.

also they aren't actually open source? Only the weights are open source?

This is probably the best explanation I've seen so far and really helped me actually understand what it means when we talk about "weights" for LLMs.

Just stay far, far away from their forums.

I'm not too informed about DeepSeek. Is it real open-source, or fake open-source?