Such an interesting and weird life to have lived.
-
Such an interesting and weird life to have lived. Academically, he lived to see almost his entire body of work thoroughly disrupted by LLMs, specifically the concept of an universal grammar/ and the idea of an 'innate language acquisition device'. I was in a live stream with him about 2 months or so after the first beta models from open AI had started to poke their way into the main-stream. It felt kind of sad, because its like, he was obviously very long in the tooth even then, but when your entire academic career is based on like "one thing", as so many scientists and philosophers careers are, when that 'one thing' ends up being demonstrably false, it seems kind of.. soul crushing? I saw it once before when the revised genomic classification of plants was being released and one of my professors (who i think was like, 1-2 years out from retirement), watched his life time body of work of taxonomic classification get "yeah nah dawged" by the revised genomic taxonomy. I'll say he was not the most engaged instructor and actually insisted on teaching us wrong, which was super aggravating.
I still think Chomskys political work stands apart as a life well lived.
-
Such an interesting and weird life to have lived. Academically, he lived to see almost his entire body of work thoroughly disrupted by LLMs, specifically the concept of an universal grammar/ and the idea of an 'innate language acquisition device'. I was in a live stream with him about 2 months or so after the first beta models from open AI had started to poke their way into the main-stream. It felt kind of sad, because its like, he was obviously very long in the tooth even then, but when your entire academic career is based on like "one thing", as so many scientists and philosophers careers are, when that 'one thing' ends up being demonstrably false, it seems kind of.. soul crushing? I saw it once before when the revised genomic classification of plants was being released and one of my professors (who i think was like, 1-2 years out from retirement), watched his life time body of work of taxonomic classification get "yeah nah dawged" by the revised genomic taxonomy. I'll say he was not the most engaged instructor and actually insisted on teaching us wrong, which was super aggravating.
I still think Chomskys political work stands apart as a life well lived.
wrote on last edited by [email protected]This comment is global-warming-denialism levels of stupid. I’m honestly shocked.
LLM’s have no such implications for the field of linguistics. They’re barely relevant at all.
Do I really need to point out that human beings do not learn language the way LLMs “learn” language? That human beings do not use language the way LLM’s “use” language? Or that human beings are not mathematical models. Not even approximately. I fucking hate this timeline.
-
This comment is global-warming-denialism levels of stupid. I’m honestly shocked.
LLM’s have no such implications for the field of linguistics. They’re barely relevant at all.
Do I really need to point out that human beings do not learn language the way LLMs “learn” language? That human beings do not use language the way LLM’s “use” language? Or that human beings are not mathematical models. Not even approximately. I fucking hate this timeline.
This would be true if chomskys claim was that he was simply studying human language acquisition and that machines are different, but his claim was that machines can't learn human languages because they don't have some intuitive innate grammar.
Saying an llm hasn't learned language becomes harder and harder the more you talk to it and the more it starts walking like a duck and quacking like a duck. To make that claim you'll need some evidence to counter the demonstrable understanding the llm displays. Chomsky in his nytimes response just gives his own unprovable theories on innate grammar and some examples of questions llms "can't answer" but if you actually ask any modern llm they answer them fine.
You can define "learning" and "understanding" in a way that excludes llms but you'll end up relying upon unprovable abstract theories until you can come up with an example of a question/prompt that any human would answer correctly and llms won't to demonstrate that difference. I have yet to see any such examples. There's plenty of evidence of them hallucinating when they reach the edge of their understanding, but that is something humans do as well.
Chomsky is still a very important figure and his work on politics with manufacturing consent is just as relevant as when it was written over 20 years ago. His work on language though is on shaky grounds and llms have made it even shakier.
-
This would be true if chomskys claim was that he was simply studying human language acquisition and that machines are different, but his claim was that machines can't learn human languages because they don't have some intuitive innate grammar.
Saying an llm hasn't learned language becomes harder and harder the more you talk to it and the more it starts walking like a duck and quacking like a duck. To make that claim you'll need some evidence to counter the demonstrable understanding the llm displays. Chomsky in his nytimes response just gives his own unprovable theories on innate grammar and some examples of questions llms "can't answer" but if you actually ask any modern llm they answer them fine.
You can define "learning" and "understanding" in a way that excludes llms but you'll end up relying upon unprovable abstract theories until you can come up with an example of a question/prompt that any human would answer correctly and llms won't to demonstrate that difference. I have yet to see any such examples. There's plenty of evidence of them hallucinating when they reach the edge of their understanding, but that is something humans do as well.
Chomsky is still a very important figure and his work on politics with manufacturing consent is just as relevant as when it was written over 20 years ago. His work on language though is on shaky grounds and llms have made it even shakier.
Do you really think Chomsky’s UG hypothesis from half a century ago was formulated to deny that some dumb mathematical model would be able to simulate human speech?
You have a sentence that reads something like “the more you talk to LLM’s the harder it is to deny that they can use and understand language.”
You might as well say that the longer you stare at a printed painting, the harder it is to deny that printers make art. LLM’s do not “understand” their outputs or their inputs. If we feed them nonsense, they output nonsense. There’s no underlying semantics whatsoever. An LLM is a mathematical model.
I know it looks like magic, but it’s not actually magic. And even if it were, it would have nothing to do with linguistics, which is concerned with how humans, not computers, understand and manipulate language. This whole ridiculous conversation is a non-sequitur.
