studies show a clear trend – output is up (more code, more commits, bigger diffs), but outcomes don’t reflect that trend. If anything, the average team is taking longer to ship worse software
output is up (more code, more commits, bigger diffs)
We’ve known measuring output by lines of code is counterproductive for a long time.
Has management known that?
A lot of problems seem to be downstream from “management are idiots and assholes”
I’m sure there were pockets but it legitimately seemed like that dragon had been slain until recently. Trying to assign more meaning to scrum points has been in vogue for a while though.
My team assigns both hours and points to tasks. I’ve never seen the points used for anything but they still spend time on it.
That’s… an interesting approach.
At any minimally competent company they are aware.
However my company mostly has former engineers as engineering managers.
One study found a significant correlation between confidence in AI output and belief in the paranormal.
💀 💀 💀
That’s HILLARIOUS
Lol that’s funny. My belief in having a divinely created soul is exactly why I think humans can’t be replaced by these supercharged drunken parrots. :O
…But yeah this is probably referring to the ones who, under the right conditions, will start to sense “ghosts in the machine” and go down the rabbit hole to chatbot psychosis…
I am 100% genuinely not trying to start a fight, this a legitimate question to which I really would like to know the answer.
What is the difference between a divine creation and the "ghost in the machine "?
I will give context , faith is genuinely confusing to me.
I have no issue with personal faiths unless you are trying to force everyone to have the same faith as you.
I consider anything above small scale organisation of religion to be the worst thing that can happen to faith, because people are people and power corrupts.
I suppose my real question is , for a concept that prohibits proof as part of it’s definition, how do you determine that one faith (or system) is better than another?
I’m not the person you responded to, but I can try to take a stab at that…
for a concept that prohibits proof as part of it’s definition, how do you determine that one faith (or system) is better than another?
Objectively, you don’t. But people’s experiences go beyond the objective, and we all have our subjective experiences too. Faith is how we make sense of those.
The question then isn’t, which one is better, but whose other experiences and faiths do we relate to? We build friendships and alliances based on those. Not because they’re better, per se, but because we feel comfortable or safe with them.
From that perspective it makes a lot of sense that we value other humans, because they can make us feel understood and appreciated, and we can exchange ideas with them which in turn refines our faith. Now, some people get tricked into thinking you can do the same with machines, but well, I think you can guess where I too stand on that idea.
That was a very insightful answer. Well said! Thank you very much for replying. :)
I will have to contemplate a little bit, and respond to the question myself as well.
i have posted a response to the above comment that might contain relevant information.
Thank you for taking the time to respond.
That is a well thought out and helpful answer, that’s useful to me in understanding come aspects of faith in general, thank you.
Before i continue i want to highlight the differences i see between:
- LLM (Large Language Models) - The trending technology that is a smaller part of a general field. - it's what you are interacting with using ChatGPT, Claude etc - AI (Artifical Intelligence) - The general field, which includes LLM's but also Neural Networks, certain kinds of algorithms and more. - AGI (Artificial **General** Intelligence) - The common scientific term for what most people think of when they hear "AI". - roughly and borderline inaccurately: - "a machine intellect of the same type as a human, matching or exceeding common human capability"I get that faith, mysticism etc are how some of us deal with subjective ( from our point of view ) phenomena . I’d imagine this is why gods for things like storms or seasons etc come about.
As i said , I have no issue with any of that unless it starts being forced on people, i even vaguely understand how people could come to that conclusion and i’m not pretending to have any better answers.
My original question was more about this part:
Now, some people get tricked into thinking you can do the same with machines, but well, I think you can guess where I too stand on that idea.
Are you saying that the subjective experiences with the LLM’s are the trick or the feeling of understanding and appreciation with the machine ?
Some peoples experiences with LLM’s (perhaps what they consider to be an AGI?) sounds like they’d fall exactly in to the type of subjective phenomena that would bring about faith as a means of processing.
That some people seem to think that that kind of faith is any less valid than a guardian deity or a wind god is where I’m struggling.
I don’t understand how people are making that kind of distinction, unless it’s not something based in reasoning?
I want to make clear i don’t think LLM’s are sapient or sentient at this point, though i can’t personally disprove it and am not against being convinced otherwise should sufficient rationale be provided.
There’s a lot of levels at which I could reply to this comment, because it’s quite a philosophical topic, but let me clarify what I meant at the end of my last comment, before writing a wall of text here :)
As I said in my previous comment, faith is largely a means to make sense of our subjective experiences. We can refine our faith by talking about our experiences with other people.
So the most basic fallacy I was alluding to, from people who believe you can refine your faith by talking with an LLM is this: An LLM has no subjective experiences, because it doesn’t experience anything at all. This is one way how using the output from an LLM as a source of inspiration for your faith sets you up for psychosis. You start believing things that have no basis in reality or anyone’s genuine experiences. Rather, you may start believing in machine-generated approximations from which coherence has been lost.
But it seems your understanding was slightly different, in that people might develop their own sense of faith from experiences with an LLM itself rather than through the exchange of presumed experiences with the LLM. Do I understand correctly that you think some people attribute godlike qualities to the LLM, maybe through the presumption that it is an AGI? That would be a bit of a different discussion indeed :)
So the most basic fallacy I was alluding to, from people who believe you can refine your faith by talking with an LLM is this: An LLM has no subjective experiences, because it doesn’t experience anything at all. This is one way how using the output from an LLM as a source of inspiration for your faith sets you up for psychosis. You start believing things that have no basis in reality or anyone’s genuine experiences. Rather, you may start believing in machine-generated approximations from which coherence has been lost.
Reasonable, though it could be argued they are full of subjective experiences, just all mixed together.
But i expect it’s probably more about the coherence of the experience that provides the meaning in this context.
Coherence as in , one coherent explanation of a subjective experience by a person rather than a plausible sounding experience made up of many bits of experiences from a corpus of data.
But it seems your understanding was slightly different, in that people might develop their own sense of faith from experiences with an LLM itself rather than through the exchange of presumed experiences with the LLM. Do I understand correctly that you think some people attribute godlike qualities to the LLM, maybe through the presumption that it is an AGI? That would be a bit of a different discussion indeed :)
Not exactly, i think there are two different things here from my side.
The first:
I think people might attribute person like qualities to an LLM, the “ghost in the machine” as was mentioned earlier.
I wasn’t trying to imply someone who knows what an AGI is supposed to be would use that knowledge to assign godlike attributes to an LLM (a “god in the machine”).
Though , as a concept that would also be a valid example of “belief in the absence of proof”.
I don’t expect most people to know the terminology or have even an idea of what’s happening behind the scenes.
The second:
The part i’m struggling with is why “faith” in something like ability for meaning to come from LLM generated text (god-like or person-like) is any less acceptable than “faith” derived from perceived meaning in seemingly random (based on the knowledge of the time) events, like thunderstorms or plagues.
They would both cover the idea of belief in something in the absence of proof, so how would someone decide one is less valid than the other.
Right, gotcha. So, a few things here:
Reasonable, though it could be argued they are full of subjective experiences, just all mixed together.
This argument wouldn’t really make sense to me, because an experience and the words describing said experience are clearly not the same. An LLM only possesses the later, but the actual experience is something conscious that an LLM does not have.
Words are approximations, and often poor ones at that. For us humans, we can use them to describe experiences so that hopefully they bring understanding in the recipient, but whether the recipient truly understands and whether they can relate to the experience, also depends on their own experiences and their empathic abilities. None of that is at work inside an LLM.
But indeed, thinking that an LLM does have such abilities would be an example of attributing person-like qualities to it.
Now, if a person does develop a sense of faith from LLM output, is that more or less valid than them developing a sense of faith through a thunderstorm? That depends on the faith, I guess.
If someone truly believes the LLM to be person-like (or godlike) that would be a nonsensical faith, as much as someone believing that firestorms are the wrath of god, for instance.
If someone believes that an LLM-generated text provided them with a critical insight that turned their life around, and therefore LLMs are divinely inspired, it’s as harmless as someone believing that a thunderbolt that struck nearby gave them the necessary wakeup call that turned their life around, and therefore thunderstorms are guided by the divine.
Of course in either case, if they start to revere LLMs/thunderstorms above other aspects of reality, it becomes problematic.
Personally, as a more spiritually-inclined person, I think we should be able to say that both thunderstorms and LLMs are awe-inspiring, and I think both can, in theory, be a vessel for the divine. But ultimately, neither are divine in and of themselves.
So isn’t there a difference between faith based on LLMs vs. faith based on firestorms? I think it really depends on the case, but there’s still an important distinction: A firestorm doesn’t pretend to be human, or doesn’t try to trick people into pretending it has humanlike qualities or experiences. So generally speaking, people tend not to develop weird faiths based on thunderstorms. But LLMs are, by design, imposters pretending to have humanlike qualities. This is why I think we tend to be dismissive of people believing in things told by LLMs, because it’s so easy to fall into the idea of ascribing real humanlike qualities to them and developing psychotic thoughts from there.
This doesn’t seem to cover there is also no LLM that doesn’t plagiarize, or where the training data appears to be compatible with such behavior (e.g. CC0). Now I don’t know what that means legally, but morally it seems to be tossing away other project’s licensing and I think for FOSS as a whole that’s no good.
Also something worth reiterating: https://machinelearning.apple.com/research/illusion-of-thinking LLMs apparently can’t do basic logical reasoning. Even a junior coder can do that. I’m always surprised anybody would let LLMs near their code, at all.
I recommed you reading this
It summarizes really good not only the moral, but also the legal problems of AI, vibecoding and “AI-assisted/AI-boosted” programming/engineering/development.
This aligns with my experience, largely. Of course it’s still my job to maximize LLM effectiveness within my organization. Which is a delicate balancing act to protect my teams from overeager executive leadership looking for huge gains.
My own summary is that AI can be an accelerator, but the harder you lean into it, the worse outcomes will be. No matter how much code is written, you still need actual human minds to understand it and they can only handle so much volume before getting overwhelmed.
Also, if AI gives you 20% productivity gains, but that 20% goes into playing with AI trying to get more, you haven’t really gained anything. Usage needs to be standardized rather than developers constantly negotiating with AI trying to coax out better outcomes.
human minds … can only handle so much volume before getting overwhelmed.
One might even consider flourishing employees as opposed to not-burned-out ones.
This is a tale older than AI. Most of the AI productivity pushes I struggle to get adopted fail not because of AI bad or its too hard to do. They fail because of a broken CI/CD pipeline. They fail because some team thinks their process is sacred and unique.
LLMs cannot distinguish between recent and out-of-date information in the context, and information in the model itself, learned during training (“dominant priors”), can often “outweigh” information we give it
LLM inference is more accurate when we give them examples (demonstrations) rather than just describing what we want.
Deep neural networks, including LLMs, struggle to learn patterns with long-range dependencies, at any scale of model. They will always be “driving in fog”, with local, short-range probabilities crowding out long-range ones. In case you were wondering why they suck at the “big picture” – probabilistically, it’s a blur.
I’d guess that what all this stuff adds up to is, sustainable use of LLMs as a coding tool for nontrivial projects calls for an entirely reworked set of software development practices to conform to its limitations effectively, but the people in charge really really want and believe it to be a drop-in efficiency boost, and a big mess results. This reminds me a lot of the articles and arguments I’ve read over the years about low level vs high level programming languages and frameworks. Probably will play out a similar way.
I could’ve told them this, tbh.
LLMs can’t tell the difference between code and comments in some cases. Older code bases that have had a number of hands touching it over the years have a lot of commented out code and dead methods. LLM sees that as good live code. Also there will be comments like ‘this is a stupid way to do this, but I don’t have time to fix it right now’. LLM has no sense of humor in this case.
And sure. I should clean all this up before hand, but who has time for that? Project has one, part time developer working on it now.Hehehehe
LLMs struggle with negation. Telling them not to do something can often have the same effect as telling them to do it.
The future looks… unreliable.
Models may get more powerful, but not significantly more reliable. This it folks – work with what you’ve got!
We all know that true AGIs becoming smarter than humans seems inevitable, but that could be like a hundred years from now, if ever. What’s unclear is what will happen two years from now, involving matters having little to do with the technology & what it is capable of and instead more to do with the economy and what jobs will be available then.
15 - 30 years of pain followed by the end of life as we know it those who adapt may thrive but will continue to be exploited.
Tbf to LLM manufacturers, the end of life as we know it was coming either way.
Someone’s ITIL certified
studies show a clear trend – output is up (more code, more commits, bigger diffs), but outcomes don’t reflect that trend. If anything, the average team is taking longer to ship worse software
&
burning out sooner,
suffocating high pace churn,
debugging bad code.
