Didnt it turn out that they used 10000 nvidia cards that had the 100er Chips, and the "low level success" is a lie?

There's actually a typo, i wrote "relies or bullshit" instead of " on bullshit"

I would say that in comparison to the standards used for top ML conferences, the paper is relatively light on the details. But nonetheless some folks have been able to reimplement portions of their techniques.

ML in general has a reproducibility crisis. Lots of papers are extremely hard to reproduce, even if they're open source, since the optimization process is partly random (ordering of batches, augmentations, nondeterminism in GPUs etc.), and unfortunately even with seeding, the randomness is not guaranteed to be consistent across platforms.

You’re right. That’s it!

This guy found the original.

https://lemmy.world/comment/15023547

It was originally about Napster / Gnutella.

Definitions are tricky, and especially for terms that are broadly considered virtuous/positive by the general public (cf. "organic") but I tend to deny something is open source unless you can recreate any binaries/output AND it is presented in the "preferred form for modification" (i.e. the way the GPLv3 defines the "source form").

A disassembled/decompiled binary might nominally be in some programming language--suitable input to a compiler for that langauge--but that doesn't actually make it the source code for that binary because it is not in the form the entity most enabled to make a modified form of the binary (normally or original author) would prefer to make modifications.

immediately fled to Russia with troves of American secrets

Russia wasn't his final destiny. He was en route to somewhere else when his passport was revoked.

also they aren't actually open source? Only the weights are open source?

This is probably the best explanation I've seen so far and really helped me actually understand what it means when we talk about "weights" for LLMs.

Just stay far, far away from their forums.

I'm not too informed about DeepSeek. Is it real open-source, or fake open-source?

It’s semi-open, not fully open source as what is typically thought of.