cross-posted from: https://lemmy.ml/post/2811405
"We view this moment of hype around generative AI as dangerous. There is a pack mentality in rushing to invest in these tools, while overlooking the fact that they threaten workers and impact consumers by creating lesser quality products and allowing more erroneous outputs. For example, earlier this year America’s National Eating Disorders Association fired helpline workers and attempted to replace them with a chatbot. The bot was then shut down after its responses actively encouraged disordered eating behaviors. "
The real issue is people need to realize how LLMs work. It’s just a really good next word generator that sounds plausible to a human. Accuracy and truth isn’t part of consideration for the most part. The AI doesn’t even see words, it just breaks words down to numbers and treats it like a giant math problem.
It’s an amazing tool that will massively boost productivity, but people need to know its limitations and what it’s actually capable of. That’s where the hype is overblown.
Ironically, I think you also are overlooking some details about how LLMs work. They are not just word generators. Stuff is going on inside those neural networks that we’re still unsure of.
For example, I read about a study a little while back that was testing the mathematical abilities of LLMs. The researchers would give them simple math problems like “2+2=” and the LLM would fill in 4, which was unsurprising because that equation could be found in the LLM’s training data. But as they went to higher numbers the LLM kept giving mostly correct results, even when they knew for a fact that the specific math problem being presented wasn’t in the training data. After training on enough simple addition problems the LLM had actually “figured out” some of the underlying rules of math and was using those to make its predictions.
Being overly dismissive of this technology is as fallacious as overly hyping it.
I think this is downplaying what LLMs do. Yeah, they are not the best at doing things in general, but the fact that they were able to learn the structure and semantic context of language is quite impressive, even if it doesn’t know what the words converted into tokens actually mean. I suspect that we will be able to use LLMs as one part of a full digital “brain”, with some model similar to our own prefrontal cortex calling the LLM (and other things like vision model, sound model, etc.) and using its output to reason about a certain task and take an action. That’s where I think the hype will be validated, is when you put all these parts we’ve been working on together and Frankenstein a new and actually intelligent system.