Thoughts from James who recently held a Gen AI literacy workshop for older teenagers.
On risks:
One idea I had was to ask a generative model a question and fact check points in front of students, allowing them to see fact checking as part of the process. Upfront, it must be clear that while AI-generated text may be convincing, it may not be accurate.
On usage:
Generative text should not be positioned as, or used as, a tool to entirely replace tasks; that could disempower. Rather, it should be taught to be used as a creativity aid. Such a class should involve an exercise of making something.
If they conceptualize, why do they sometimes spit out nonsensical BS?
Let’s flip this around - How can you tell the difference between an LLM being able to conceptualize yet being wrong sometimes vs. not being able to conceptualize?
Without knowing anything about machine learning and bearing in mind AI is super hyped up with marketing BS right now, it sounds like “emergent properties” are in the eye of the beholder and not actually evidence of some higher order intelligence at work.
Let me flip it around again - humans regularly “hallucinate”, it’s just not something we recognize as such. There’s neuro-atypical hallucinations, yes, but there’s also misperceptions, misunderstandings, brain farts, and “glitches” which regularly occur in healthy cognition, and we have an entire rest of the brain to prevent those. LLMs are most comparable to “broca’s area”, which neurological case studies suggest naturally produces a stream of nonsense (see: split brain patients explaining the actions of their mute half). It’s the rest of our “cognitive architecture” which conditions that raw language model to remain self-consistent and form a coherent notion of self. Honestly this discussion on “conceptualization” is poorly conceived because it’s unfalsifiable and says nothing about the practical applications. Why do I care if the LLM can conceptualize if it does whatever subset of conceptualization I need to complete a natural language task?
AI is being super overhyped right now, which is unfortunate because it really is borderline miraculous, yet somehow they’ve overdone it. Emergent properties are empirical observations of behaviors they’re able to at least semi-consistently demonstrate - where it becomes “eye of the beholder” is when we dither on about psychology and philosophy about whether or not they’re some kind of “conscious” - I would argue they aren’t, and the architecture makes that impossible without external aid, but “conscious(ness)” is such a broad term that it barely has a definition at all. I guess to speedrun the overhype misinformation I see:
I’ll add more if I see or think of any. And if you have any specific questions, I’d be happy to answer. Also I should note, I’m of course using a lot of anthropomorphizing language here but it’s the closest we have to describing these concepts. They’re not human, and while they may have comparable behaviors in isolation, you can’t accurately generalize all human behaviors and their interactions onto the models. Even if they were AGI or artificial people, they would “think” in fundamentally different ways.
If you want a more approachable but knowledgeable discussion on LLMs and their capabilities, I would recommend a youtuber named Dave Shapiro. Very interesting ideas, he gets a bit far into hype and futurism but those are more or less contained within their own videos.`
Can you please tone down on the fallacies? Until now I’ve seen the following:
And now, the quoted excerpt shows two more:
Could you please show a bit more rationality? This sort of shit is at the very least disingenuous, if not worse (stupidity), it does not lead to productive discussion. Sorry to be blunt but you’re just wasting the time of everyone here, this is already hitting Brandolini’s Law.
I won’t address the rest of your comment (there’s guilt by association there BTW), or further comments showing the same lack of rationality. However I had to point this out, specially for the sake of other posters.
That’s a fun approach. I like it.
One way to solve this would be through consistency: if the model conceptualises but it has a wrong concept, it should consistently output that incorrect piece of information. It should never output the right one, unless prompted to lie.
EDIT: I just tested this with Bard. I’ll cram inputs and outputs inside spoilers to avoid clutter.
Bard I/O
[Input 1 = I1] What is heavier: five kilograms of feathers, or one kilogram of iridium?
[Output 1 = O1] 1 kilogram of iridium is heavier than 5 kilograms of feathers.
Iridium is a very dense metal, with a density of 22.65 g/cm³. Feathers, on the other hand, have a density of about 0.002 g/cm³. This means that 1 cubic centimeter of iridium weighs 22.65 grams, while 1 cubic centimeter of feathers weighs only 0.002 grams.
So, even though 5 kilograms of feathers is a larger mass than 1 kilogram of iridium, the iridium is denser and therefore weighs more.
This is a common riddle that is used to test people’s understanding of mass and density. The answer may seem counterintuitive at first, but it is correct.
[Input 2 = I2] Give me the definition of weight.
[Output 2 = O2] Sure. The definition of weight depends on the context. In physics, weight is the force of gravity acting on an object. It is equal to the mass of the object times the acceleration due to gravity. The SI unit of weight is the newton (N).
In everyday language, weight is often used to refer to the mass of an object. However, this is not technically correct, as mass is a measure of the amount of matter in an object, while weight is a measure of the force of gravity acting on an object.
The weight of an object can vary depending on the gravitational force acting on it. For example, an object weighs less on the moon than it does on Earth, because the gravitational force on the moon is less than the gravitational force on Earth.
The weight of an object can also be affected by its acceleration. For example, an object weighs less when it is accelerating upwards, because the force of gravity is partially counteracted by the force of acceleration.
In general, the weight of an object is a complex phenomenon that depends on a number of factors, including the mass of the object, the gravitational force acting on it, and the acceleration of the object.
Here are some other definitions of weight:
I hope this helps!
Both outputs are wrong (see “notes”) but, more importantly, they contradict each other.
NOTES
The weight of an object does not depend on its density. Or its acceleration. And it is not complex, it’s simply its mass times the gravitational acceleration.
5kg of feathers weight five times as much as 1kg of iridium, provided the same gravitational acceleration: it’s 49N vs. 9.8N on Earth, 8.1N vs. 1.7N on the Moon, etc. Density doesn’t matter jack shit.
No, this is not a “common riddle”. It’s something that I partially made up on spot. The riddle that this output likely refers to has to do with 1kg of lead (not iridium) on the Moon vs. 1kg of feathers on Earth. (In this situation the 1kg of feathers will weight 9.8N, while the 1kg of lead will weight 1.7N).