Majority of AI Researchers Say Tech Industry Is Pouring Billions Into a Dead End
-
Optimizing AI performance by “scaling” is lazy and wasteful.
Reminds me of back in the early 2000s when someone would say don’t worry about performance, GHz will always go up.
It always wins in the end though. Look up the bitter lesson.
-
I agree that it's editorialized compared to the very neutral way the survey puts it. That said, I think you also have to take into account how AI has been marketed by the industry.
They have been claiming AGI is right around the corner pretty much since chatGPT first came to market. It's often implied (e.g. you'll be able to replace workers with this) or they are more vague on timeline (e.g. OpenAI saying they believe their research will eventually lead to AGI).
With that context I think it's fair to editorialize to this being a dead-end, because even with billions of dollars being poured into this, they won't be able to deliver AGI on the timeline they are promising.
Part of it is we keep realizing AGI is a lot more broader and more complex than we think
-
I remember listening to a podcast that’s about explaining stuff according to what we know today (scientifically). The guy explaining is just so knowledgeable about this stuff and he does his research and talk to experts when the subject involves something he isn’t himself an expert.
There was this episode where he kinda got into the topic of how technology only evolves with science (because you need to understand the stuff you’re doing and you need a theory of how it works before you make new assumptions and test those assumptions). He gave an example of the Apple visionPro being a machine that despite being new (the hardware capabilities, at least), the algorithm for tracking eyes they use was developed decades ago and was already well understood and proven correct by other applications.
So his point in the episode is that real innovation just can’t be rushed by throwing money or more people at a problem. Because real innovation takes real scientists having novel insights and experiments to expand the knowledge we have. Sometimes those insights are completely random, often you need to have a whole career in that field and sometimes it takes a new genius to revolutionize it (think Newton and Einstein).
Even the current wave of LLMs are simply a product of the Google’s paper that showed we could parallelize language models, leading to the creation of “larger language models”. That was Google doing science. But you can’t control when some new breakthrough is discovered, and LLMs are subject to this constraint.
In fact, the only practice we know that actually accelerates science is the collaboration of scientists around the world, the publishing of reproducible papers so that others can expand upon and have insights you didn’t even think about, and so on.
There's been several smaller breakthroughs since then that arguably would not have happened without so many scientists suddenly turning their attention to the field.
-
Imo to make an ai that is truly good at everything we need to have multiple ai all designed to do something different all working together (like the human brain works) instead of making every single ai a personality-less sludge of jack of all trades master of none
Lots of people think this. They keep turning out wrong. Look up the bitter lesson
-
I like my project manager, they find me work, ask how I'm doing and talk straight.
It's when the CEO/CTO/CFO speaks where my eyes glaze over, my mouth sags, and I bounce my neck at prompted intervals as my brain retreats into itself as it frantically tosses words and phrases into the meaning grinder and cranks the wheel, only for nothing to come out of it time and time again.
Find a better C-suite
-
This post did not contain any content.
-
Says the country where every science textbook is half science half conversion tables.
Not even close.
Yes, one half is conversion tables. The other half is scripture disproving Darwinism.
-
As an experienced software dev I'm convinced my software quality has improved by using AI. More time for thinking and less time for execution means I can make more iterations of the design and don't have to skip as many nice-to-haves or unit tests on account of limited time. It's not like I don't go through every code line multiple times anyway, I don't just blindly accept code. As a bonus I can ask the AI to review the code and produce documentation. By the time I'm done there's little left of what was originally generated.
If a bot can develop your software better than you then you're a shit software dev
-
I am indeed getting more time off for PD
We delivered on a project 2 weeks ahead of schedule so we were given raises, I got a promotion, and we were given 2 weeks to just do some chill PD at our own discretion as a reward. All paid on the clock.
Some companies are indeed pretty cool about it.
I was asked to give some demos and do some chats with folks to spread info on how we had such success, and they were pretty fond of my methodology.
At its core delivering faster does translate to getting bigger bonuses and kickbacks at my company, so yeah there's actual financial incentive for me to perform way better.
You also are ignoring the stress thing. If I can work 3x better, I can also just deliver in almost the same time, but spend all that freed up time instead focusing on quality, polishing the product up, documentation, double checking my work, testing, etc.
Instead of scraping past the deadline by the skin of our teeth, we hit the deadline with a week or 2 to spare and spent a buncha extra time going over everything with a fine tooth comb twice to make sure we didn't miss anything.
And instead of mad rushing 8 hours straight, it's just generally more casual. I can take it slower and do the same work but just in a less stressed out way. So I'm literally just physically working less hard, I feel happier, and overall my mood is way better, and I have way more energy.
Are you a software engineer? Without doxxing yourself, do you think you could share some more info or guidance? I've personally been trying to integrate AI code gen into my own work, but haven't had much success.
I've been able to ask ChatGPT to generate some simple but tedious code that would normally require me read through a bunch of documentation. Usually, that's a third party library or a part of the standard library I'm not familiar with. My work is mostly Python and C++, and I've found that ChatGPT is terrible at C++ and more often than not generates code that doesn't even compile. It is very good at generating Python by comparison, but unfortunately for me, that's only like 10% of my work.
For C++, I've found it helpful to ask misc questions about the design of the STL or new language features while I'm studying them myself. It's not actually generating any code, but it definitely saves me some time. It's very useful for translating C++'s "standardese" into english, for example. It still struggles generating valid code using C++20 or newer though.
I also tried a few local models on my GPU, but haven't had good results. I assume it's a problem with the models I used not being optimized for code, or maybe the inference tools I tried weren't using them right (oobabooga, kobold, and some others I don't remember). If you have any recommendations for good coding models I can run locally on a 4090, I'd love to hear them!
I tried using a few of those AI code editors (mostly VS Code plugins) years ago, and they really sucked. I'm sure things have improved since then, so maybe that's the way to go?
-
This post did not contain any content.
Why won't they pour billions into me? I'd actually put it to good use.
-
Why won't they pour billions into me? I'd actually put it to good use.
I'd be happy with a couple hundos.
-
I'd be happy with a couple hundos.
I'd be happy with a big tiddy goth girl. Jealous of your username btw.
-
I have been shouting this for years. Turing and Minsky were pretty up front about this when they dropped this line of research in like 1952, even lovelace predicted this would be bullshit back before the first computer had been built.
The fact nothing got optimized, and it still didn't collapse, after deepseek? kind of gave the whole game away. there's something else going on here. this isn't about the technology, because there is no meaningful technology here.
Companies aren't investing to achieve AGI as far as I'm aware, that's not the end game so I this title is misinformation. Even if AGI was achieved it'd be a happy accident, not the goal.
The goal of all these investments is to convince businesses to replace their employees with AI to the maximum extent possible. They want that payroll money.
The other goal is to cut out all third party websites from advertising revenue. If people only get information through Meta or Google or whatever, they get to control what's presented. If people just take their AI results at face value and don't actually click through to other websites, they stay in the ecosystem these corporations control. They get to sell access to the public, even more so than they do now.
-
If a bot can develop your software better than you then you're a shit software dev
That's not what is happening. The bot writes code and then I tell it what to change until it's close enough, then I make the final touches myself. It's like having a junior programmer do the grunt work for you.
-
This post did not contain any content.
It doesnt matter if they reach any end result, as long as stocks go up and profits go up.
Consumers arent really asking for AI but its being used to push new hardware and make previous hardware feel old. Eventually everyone has AI on their phone, most of it unused.
-
-
I have been shouting this for years. Turing and Minsky were pretty up front about this when they dropped this line of research in like 1952, even lovelace predicted this would be bullshit back before the first computer had been built.
The fact nothing got optimized, and it still didn't collapse, after deepseek? kind of gave the whole game away. there's something else going on here. this isn't about the technology, because there is no meaningful technology here.
-
This post did not contain any content.
The problem is that those companies are monopolies and can raise prices indefinitely to pursue this shitty dream because they got governments in their pockets. Because gov are cloud / microsoft software dependent. They can like raise prices 10x times in next 10 years and don't give a fuck. Spend 1 trillion on AI and say we're near over and over again and literally nobody can stop them right now.
-
It doesnt matter if they reach any end result, as long as stocks go up and profits go up.
Consumers arent really asking for AI but its being used to push new hardware and make previous hardware feel old. Eventually everyone has AI on their phone, most of it unused.
I enough researchers talk about the problems them that will eventually break through the bubble and investors will pull out.
We're at the stage of the new technology hype cycle where it crashes, essentially for this reason. I really hope it does soon because then they'll stop trying to force it down our throats in every service we use.
-
Are you a software engineer? Without doxxing yourself, do you think you could share some more info or guidance? I've personally been trying to integrate AI code gen into my own work, but haven't had much success.
I've been able to ask ChatGPT to generate some simple but tedious code that would normally require me read through a bunch of documentation. Usually, that's a third party library or a part of the standard library I'm not familiar with. My work is mostly Python and C++, and I've found that ChatGPT is terrible at C++ and more often than not generates code that doesn't even compile. It is very good at generating Python by comparison, but unfortunately for me, that's only like 10% of my work.
For C++, I've found it helpful to ask misc questions about the design of the STL or new language features while I'm studying them myself. It's not actually generating any code, but it definitely saves me some time. It's very useful for translating C++'s "standardese" into english, for example. It still struggles generating valid code using C++20 or newer though.
I also tried a few local models on my GPU, but haven't had good results. I assume it's a problem with the models I used not being optimized for code, or maybe the inference tools I tried weren't using them right (oobabooga, kobold, and some others I don't remember). If you have any recommendations for good coding models I can run locally on a 4090, I'd love to hear them!
I tried using a few of those AI code editors (mostly VS Code plugins) years ago, and they really sucked. I'm sure things have improved since then, so maybe that's the way to go?
I primarily use GPT style tools like ChatGPT and whatnot.
The key is, rather than asking it to generate code, specify that you dont want code and instead want it to help you work through the solution. Tell it to ask you meaningful questions about your problem and effectively act as a rubber duck
Then, after you've chosen a solution with it, ask it to generate code based on all the above convo.
This will typically produce way higher quality results and helps avoid potential X/Y problems.
