Over the past few years, the evolution of AI-driven tools like GitHub’s Copilot and other large language models (LLMs) has promised to revolutionise programming. By leveraging deep learning, these tools can generate code, suggest solutions, and even troubleshoot issues in real-time, saving developers hours of work. While these tools have obvious benefits in terms of productivity, there’s a growing concern that they may also have unintended consequences on the quality and skillset of programmers.
I’ve seen enough programmers blindly copypasting code from stackoverflow and other forums without thinking and never understanding the thing they just “wrote”, to know that tools like copilot won’t make programmers worse, they will allow more people to be bad programmers.
people need to read more code, play around with it, break it and fix it to become better programmers.
Hehe, good point.
people need to read more code, play around with it, break it and fix it to become better programmers.
I think AI bots can help with that. It’s easier now to play around with code which you could not write by yourself, and quickly explore different approaches. And while you might shy away from asking your colleagues a noob question, ChatGPT will happily elaborate.
In the end, it’s just one more tool in the box. We need to learn when and how to use it wisely.
I have heard the same rhetoric about IDEs, autocomplete (Intellisense, Jedi, etc.), DevOps, and frameworks. The kernel of truth across all of them is the separation between a dev and good dev. It is getting easier and easier to have something built for you using AI in your IDE in a framework that abstracts all the things away dumped into a prebuilt pipeline that deploys your artifacts for you. A dev can do that. A good dev understands the tools and knows when to dig into things.
I have yet to see a decrease in the number of good devs I meet even though IDEs slowly replaced text editors (and editors became strong enough to become IDEs). Frameworks have enabled more good devs to focus on business logic. DevOps provides solid guard rails for everything.
I don’t know if there’s an increase in the number of superficial devs. I haven’t interviewed junior dev candidates in awhile. I do know the market is flooded right now so I’d argue there might be other factors.
Also overall I do agree with the idea that letting copilot do everything for you means you don’t understand anything. Shit was the same way when cookbooks were common.
There are a LOT of superficial devs out there. You dont even have to be interviewing junior devs. Plenty of them out there at medium and senior levels. They existed before LLMs were spitting code like today, and this will undoubtedly lower the bar for bad developers to enter. It remains to be seen if this can help the gold developers in a meaningful way.
They existed before LLMs were spitting code like today, and this will undoubtedly lower the bar for bad developers to enter.
If LLMs allow bad programmers to deliver work with good enough quality to pass themselves off as good programmers, this means LLMs are fantastic value for money.
Also worth noting: programmers do learn by analysing the output of LLMs, just as the programmers of old learned by reading someone else’s code.
I think I could have states my opinion better. I think LLMs total value remains to be seen. They allow totally incompetent developers to occasionally pass as below average developers. Is that good or bad? I don’t know. What an average and excellent developer can do with LLM assistance is less clear. Certainly it can help those developers in some situations.
I think I could have states my opinion better. I think LLMs total value remains to be seen. They allow totally incompetent developers to occasionally pass as below average developers.
This is a baseless assertion from your end, and a purely personal one.
My anecdotal evidence is that the best software engineers I know use these tools extensively to get rid of churn and drudge work, and they apply it anywhere and everywhere they can.
I’m don’t disagree. Good developers use the tools to do better, but its incremental not revolutionary improvements for already competent developers.
I was hoping this might start with some actual evidence that programmers are in fact getting worse. Nope, just a single sentence mentioning “growing concern”, followed by paragraphs and paragraphs of pontification.
I don’t think it’s making devs worse, however I do think it’s significantly lowering the bar to entry to the point where people who don’t have enough knowledge to actually do the job well are becoming proceedingly common. Theoretically they should get weeded out by a good interview process but corporate be corporate
Not that my opinion is worth anything, it’s not like I have anything to back it up.
Please disregard any takes I may have
I mean, at least you acknowledge that you’re presenting an opinion. This blog post just tries to gloss over the fact that it’s pure speculation.
We’ve all read this post multiple times. Isn’t it just the “young people are lazy” that’s been going around for thousands of years?
https://historyhustle.com/2500-years-of-people-complaining-about-the-younger-generation/
At most it’s a tangent on it…
Welcome to the Internet. Pontification is all we’ve got. Now we’ve got LLMs regurgitating the old pontifications to make new ones.
I came in with your same expectations and found the same shit. Just some opinion formed on the basis of “concern”.
I’m a 10+ (cumulative) yr. experience dev. While I never used The GitHub Copilot specifically, I’ve been using LLMs (as well as AI image generators) on a daily basis, mostly for non-dev things, such as analyzing my human-written poetry in order to get insights for my own writing. And I already did the same for codes I wrote, asking for LLMs to “Analyze and comment” my code, for the sake of insights. There were moments when I asked it for code snippets, and almost every code snippet it generated was indeed working or just needing few fixes.
They’ve been becoming good at this, but not enough to really replace my own coding and analysis. Instead, they’re becoming really better for poetry (maybe because their training data is mostly books and poetry works) and sentiment analysis. I use many LLMs simultaneously in order to compare them:
- Free version of Google Gemini is becoming lazy (short answers, superficial analysis, problems with keeping context, drafts aren’t so diverse as they were before, among other problems)
- free version of ChatGPT is a bit better (can keep contexts, can issue detailed answers) but not enough (it does hallucinate sometimes: good for surrealist poetry but bad for code and other technical matters when precision and coherence matters)
- Claude is laughable hypersensitive and self-censoring to certain words independently of contexts (got a code or text that remotely mentions the word “explode” as in PHP’s
explodefunction? “Sorry, can’t comment on texts alluding to dangerous practices such as involving explosives”, I mean, WHAT?!?!) - Bing Copilot got web searching, but it has a context limit of 5 messages, so, only usable for quick and short things.
- Same about Bing Copilot goes for Perplexity
- Mixtral is very hallucination-prone (i.e. does not properly cohere)
- LLama has been the best of all (via DDG’s “AI Chat” feature), although it sometimes glitches (i.e. starts to output repeated strings ad æternum)
As you see, I tried almost all of them. In summary, while it’s good to have such tools, they should never replace human intelligence… Or, at least, they shouldn’t…
Problem is, dev companies generally focus on “efficiency” over “efficacy”, wishing the shortest deadlines while wishing some perfection. Very understandable demands, but humans are humans, not robots. We need our time to deliver, we need to cautiously walk through all the steps needed to finally deploy something (especially big things), or it’ll become XGH programming (Extreme Go Horse). And machines can’t do that so perfectly, yet. For now, LLM for development is XGH: really fast, but far from coherent about the big picture (be it a platform, a module, a website, etc).
Claude is laughable hypersensitive and self-censoring to certain words independently of contexts (…)
That’s not a problem, nor Claude’s main problem.
Claude’s main problem is that it is frequently down, unreliable, and extremely buggy. Overall I think it might be better than ChatGPT and Copilot, but it’s simply so unstable it becomes unusable.
What’s Copilot? ;)
A thing that hallucinates uncompilable code but somehow convinces your boss it’s a necessary tool.
Copilot is a tool for programmers who don’t want program.
I’ll never forget attending CS courses with a guy who got violently angry at having to write code. I assume he’s either thrilled with Copilot or in prison for attacking somebody over its failure to reliably write all of his code for him.
You write machine code?
No, you only describe what you want the compiler to write in machine code.
With copilot it’s still a description.
Sure but you’re also specifically telling it direct instructions which it will follow every time to the T, based on predetermined logic.
That is no where near how an LLM works. Furthermore, most programming languages require effort to learn. They night not be machine language, or even an assambler, but its still a skill you actually have to learn beyond speaking your native tongue.
Also one could make the argument that machine code is a “description” of what you want the CPU to do.
The skill beyond your native tongue is knowing what a db does and how to describe what your app does. Aka a designer, with design language. Good luck with a LLM getting it to do what you want with no domain specific language.
“No, no, not like that, I meant bigger…”
If the compiler produces a program that doesn’t match your description, you can debug the compiler. Can you debug an LLM?
Why wouldn’t a compiled program match your description (code)? The compiler is broken?? Compiled programs alwsys match their description(code).
So more likely your translation from idea to function is wrong.
Re-read your description, step through it slowly, what did you assume, that was wrong, or where did you add a mistake or typo? Sounds like I can do this in natural language or in Rust.
You can say that llms are not deterministic of what they produce, but that’s got nothing to do with making a programmer worse at their job.
If you can’t translate your idea into function and test its output to be what you want, then you are a bad programmer.
Copilot frequently produces results that need to be fixed. Compilers don’t do that. Anyone who uses copilot to generate code without understanding how that code works is a shit developer. The same is true of anyone who copies from stack overflow/etc without understanding what they’re copying.
You’re missing the point. If the program doesn’t do what it’s meant to its YOU that didn’t use the tools between you and metal, correctly. LLM involved or not, it’s how you’ve described it, in whatever ‘language’ you chose (natural or Rust)
The key difference is that compilers don’t fuck up, outside of the very rare compiler bug. LLMs do fuck up, quite often.
And many programmers write some pretty stupid and horrible descriptions. LLMs don’t solve this, they just allow lazy programmers to be even lazier.
Anybody that doesn’t write binary is lazy, said the compiler.
I don’t even know how to respond to this. It makes no sense at all and doesn’t really relate to or respond to my comment except it happens to use the word “lazy”, I’m guessing in reference to my comment. Good luck trying to push LLMs, not sure what your agenda really is, other than to be argumentative here. Peace.