Wondering if Modern LLMs like GPT4, Claude Sonnet and llama 3 are closer to human intelligence or next word predictor. Also not sure if this graph is right way to visualize it.
That’s literally how llma work, they quite literally are just next word predictors. There is zero intelligence to them.
It’s literally a while token is not “stop”, predict next token.
It’s just that they are pretty good at predicting the next token so it feels like intelligence.
So on your graph, it would be a vertical line at 0.
This is true if you describe a pure llm, like gpt3
However systems like claude, gpt4o and 1o are far from just a single llm, they are a blend of tailored llms, machine learning some old fashioned code to weave it all together.
Op does ask “modern llm” so technically you are right but i believed they did mean the more advanced “products”
Though i would not be able to actually answer ops questions, ai is hard to directly compare with a human.
In most ways its embarrassingly stupid, in other it has already surpassed us.
No, unfortunately you are wrong.
Gpt4 is a better version of gpt3.
The brand new one that is allegedly “unhackable” just has a role hierarchy providing rules and that hasn’t been fulled tested in the wild yet.
First, did you read even the research papers?
Secondly, none are out that are actually immune to jailbreaking lol, Where did that claim come from?
Gpt4 is just an llm. Indeed the better version of gpt3
Gpt4o and 1o (claude-sonnet possibly also) rely on the generative capacities of the gpt4 model but there is allot more going under the hood that is not simply “generate the next token”
We all agree that a pure text predictor are not at all intelligent.
The discussion at hand is wether the current frontier of ai has moved the needle up. And i still would call it pretty dumb, but moving that needle, it did. Somewhere around (x2y0.5) if i have to use the meme. Stating its (0,0) just means people aren’t interested enough to pay attention, that these aren’t just llm anymore. That’s their right but i prefer people stopped joining the discussion so uninformed.
They’re still word predictors. That is literally how the technology works
Yeah, the only question is whether human brains are also just that.
no, they are not. try showing an ai a huge number of pictures of cars from the front. Then show them one car from the side, and ask them what it is.
Show a human one picture of a car from the front, then the one from the side and ask them what it is.
What if the human had never seen or heard of anything similar to cars?
I bet it’d be confused as much as the llm.
That’s why you show him one, before asking what that same car viewed from a different angle is.
I had never seen a recumbent bike before. I only needed to see one to know and recognize one whenever I see one. Even one with a different color or make and model. The human brain definitely works differently.
You know what bicycle are though. And you’re heard of recumbent bikes or things similar to it.
If you had never heard of anything similar at all to bikes, and saw a picture of a recumbent bike from the front only, you’d probably think “ I have no fucking idea what that is”.
Idk man, weird for you to think humans can kinda learn fully about something without all the required context.
you keep missing the fact that I don’t know out of nowhere. You would have just shown me one and told me what it was. Yes of course I’d be able to tell you what it was. You just taught me. With one example.
To understand a recumbent bicycle you have to understand bicycles. To understand bicycles you have to understand wheels. You have to understand humans, and human transportation. What IS transportation. What are roads. What is a pedal. What is steering. How physics works for objects in motion. Etc etc etc etc.
You truly underestimate the amount of context and previous knowledge you need to understand even the simplest things.
lol, you got me, i definitely hadn’t thought of that.
Intelligence is a measure of reasoning ability. LLMs do not reason at all, and therefore cannot be categorized in terms of intelligence at all.
LLMs have been engineered such that they can generally produce content that bears a resemblance to products of reason, but the process by which that’s accomplished is a purely statistical one with zero awareness of the ideas communicated by the words they generate and therefore is not and cannot be reason. Reason is and will remain impossible at least until an AI possesses an understanding of the ideas represented by the words it generates.
i think the first question to ask of this graph is, if “human intelligence” is 10, what is 9? how you even begin to approach the problem of reducing the concept of intelligence to a one-dimensional line?
the same applies to the y-axis here. how is something “more” or “less” of a word predictor? LLMs are word predictors. that is their entire point. so are markov chains. are LLMs better word predictors than markov chains? yes, undoubtedly. are they more of a word predictor? um…
honestly, i think that even disregarding the models themselves, openAI has done tremendous damage to the entire field of ML research simply due to their weird philosophy. the e/acc stuff makes them look like a cult, but it matches with the normie understanding of what AI is “supposed” to be and so it makes it really hard to talk about the actual capabilities of ML systems. i prefer to use the term “applied statistics” when giving intros to AI now because the mind-well is already well and truly poisoned.
what is 9?
exactly! trying to plot this is in 2D is hella confusing.
plus the y-axis doesn’t really make sense to me. are we only comparing humans and LLMs? where do turtles lie on this scale? what about parrots?
the e/acc stuff makes them look like a cult
unsure what that acronym means. in what sense are they like a cult?
Effective Accelerationism. an AI-focused offshoot from the already culty effective altruism movement.
basically, it works from the assumption that AGI is real, inevitable, and will save the world, and argues that any action that slows the progress towards AGI is deeply immoral as it prolongs human suffering. this is the leading philosophy at openai.
their main philosophical sparring partners are not, as you might think, people who disagree on the existence or usefulness of AGI. instead, they take on the other big philosophy at openai, the old-school effective altruists, or “ai doomers”. these people believe that AGI is real, inevitable, and will save the world, but only if we’re nice to it. they believe that any action that slows the progress toward AGI is deeply immoral because when the AGI comes online it will see that we were slow and therefore kill us all because we prolonged human suffering.
That just seems like someone read about Roko’s basilisk and decided to rebrand that nightmare as the mission/vision of a company.
What a time to be alive!
I’m pretty sure most of the openai guys met on lesswrong, yeah.
They’re still much closer to token predictors than any sort of intelligence. Even the latest models “with reasoning” still can’t answer basic questions most of the time and just ends up spitting back out the answer straight out of some SEO blogspam. If it’s never seen the answer anywhere in its training dataset then it’s completely incapable of coming up with the correct answer.
Such a massive waste of electricity for barely any tangible benefits, but it sure looks cool and VCs will shower you with cash for it, as they do with all fads.
I think the real differentiation is understanding. AI still has no understanding of the concepts it knows. If I show a human a few dogs they will likely be able to pick out any other dog with 100% accuracy after understanding what a dog is. With AI it’s still just stasticial models that can easily be fooled.
I disagree here. Dogs breeds are so diverse, there’s no way you could show some pictures of a few dogs and they’d be able to pick other dogs, but also rule out other dog like creatures. Especially not with 100 percent accuracy.
for example, wolves, hyenas, and african wild dogs certainly won’t ever reach 100% consensus on dog-or-not within human groups
Sorry, really.
Hyenas aren’t closely related to dogs.You better be sorry, I was not ready to learn about it.
Thanks though…
I’d have to hand in my Canadian Passport if I weren’t, buddy.
Just goes to show, though, that even we can be fooled by things that look really similar.
Shouldn’t those be opposite sides of the same axis, not two different axes? I’m not sure how this graph should work.
It could have both abilities right?
They’re not incompatible, although I think it unlikely AGI will be an LLM. They are all next word predictors, incredibly complex ones, but that doesn’t mean they’re not intelligent. Just as your brain is just a bunch of neurons sending signals to each other, but it’s still (presumably) intelligent.
Sure, they ‘know’ the context of a conversation but only by which words are most likely to come next in order to complete the conversation. That’s all they’re trained to do. Fancy vocabulary and always choosing the ‘best’ word makes them really good at appearing intelligent. Exactly like a Sales Rep who’s never used a product but knows all the buzzwords.
deleted by creator
can you give an example of any third data point such as a rock or a chicken
Wondering if Modern LLMs like GPT4, Claude Sonnet and llama 3 are closer to human intelligence or next word predictor.
They are good at sounding intelligent. But, LLMs are not intelligent and are not going to save the world. In fact, training them is doing a measurable amount of damage in terms of GHG emissions and potable water expenditure.
Are you interested in this from a philosophical perspective or from a practical perspective?
From a philosophical perspective:
It depends on what you mean by “intelligent”. People have been thinking about this for millennia and have come up with different answers. Pick your preference.
From a practical perspective:
This is where it gets interesting. I don’t think we’ll have a moment where we say “ok now the machine is intelligent”. Instead, it will just slowly and slowly take over more and more jobs, by being good at more and more tasks. And just so, in the end, it will take over a lot of human jobs. I think people don’t like to hear it due to the fear of unemployedness and such, but I think that’s a realistic outcome.
A remarkable paper has just come out on this topic.
I’m going to say x=7, y=10. The sum x+y is not 10, because choosing the next word accurately in a complex passage is hard. The x is 7, just based on my gut guess about how smart they are - by different empirical measures it could be 2 or 40.
