Indeed, my code has never been more rich in unit tests now that coding assistants make them so easy to add. If the coding assistant subsequently does something weird that changes program behaviour it knows immediately.
Indeed, my code has never been more rich in unit tests now that coding assistants make them so easy to add
Me as well. A year ago there was significant trouble with those unit tests not adding much value but that has improved a lot over the past several model updates. The unit tests aren’t 100% as good as they could or even should be (neither are human generated unit tests that take 10x as long to add) - but in terms of time invested vs value derived, they’re excellent value for effort.
If the coding assistant subsequently does something weird that changes program behaviour it knows immediately.
They check to make sure that the code continues producing the expected results. For example if I had some code that added up numbers, I might have a unit test that calls that function with the parameters “2” and “2” and checks to make sure the result is “4”. If something changes that makes it start returning something else then a flag is immediately raised.
Are these tests written by AI? Have you reviewed the tests? Do they cover all code paths and all the possible inputs and outputs? Do the tests have requirements cited (traceability)?
Unit tests have been around for a long time. They tended to be under-utilized in the past because they’re such a maintenance headache - changes / improvements in the production code that used to be a simple 5 minute thing can easily balloon to an hour or more of headache in necessary adjustments to perfectly good unit tests for “the old way” that happened to be impacted by the change.
Unit tests are a good way of keeping under-experienced programmers from accidentally breaking old things as they work on new things. They are VERY appropriate tools for LLM agent assisted development.
Boeing’s executives had motivated reasoning to skimp on actual engineering and give the results the exec wanted over the objections of actual engineers.
Sound familiar?
Frankly, no. LLMs are so sychophantic, there’s little evidence they would have flagged a warning that what he executives wanted was dangerous.
And no, LLMs are not just some tool. Never in human history has a hammer given it’s users psychosis.
The max issue on the Boeing planes wasn’t a software issue in that sense. The software actually worked as intended. Boeing just hid the fact that the software existed and made it purposefully hard to override, giving pilots quite the surprise when the plane didn’t operate the way they expected, especially when the angle of attack sensor wasn’t working.
But you really should be writing those tests regardless of whether or not you or an LLM ends up writing the actual code. That helps ensure that the code actually does what you think it does, and prevents bugs or unintended changes in behavior from sneaking in later when something changes
Well, depends. My thousand-line pet project does not need any tests, it needs to be feature complete and never touched again.
Bigger projects are usually ones where multiple people are contributing and having code that practically nobody wrote there is a huge problem on it’s own in my opinion, even if it passes a number of tests. Good tests themselves are usually harder to write than the code they are testing, I am not sure if delegating the actual fun part of development to an LLM and doing “chores” manually is a good idea.
instead of writing what you want directly you are writing tests for it.
LLMs write pretty good requirements, specifications, code, and automated tests from chat sessions. You do need to keep an eye on what it’s writing to make sure it’s not hallucinating, but hallucinations have been reducing in practice rather dramatically across the last 12-18 months of model development. They still happen, you will always need to guard against them, but usually a quick read of what is written followed by a quick test of the product will find the worst offenders. In the end, you need to check the final product to be sure it does EXACTLY what you want, just like you always did. And, it’s still a good idea (and as possible as ever) to keep the product modularized and also check every module to ensure it works as you want. How small you make those modules has always been a matter of preference / dictated by levels of concern, etc.
If you are generating tests with an LLM, it’s not deterministic.
Also, have you actually read anything an LLM wrote that’s longer than a hundered words? It’s usually immediately obvious. In the open source world things I keep seeing are exchanges like this one. And Niri is a fairly good case with it’s maintainer actually reading the PRs and catching all the bullshit that an LLM spits out. Very often nowadays people will also do reviews with an LLM and this sort of stuff will just accumulate in the codebase with nobody to reject it.
But even if it was as useful as big tech paints it to be, I’ll always be against using it based purely on morals. There is an enormous number of issues in that regard, most importatnly LLMs literally kill people both directly and indirectly. Condoning the adoption of this technology is either being ignorant or misanthropic.
If you are generating tests with an LLM, it’s not deterministic.
Once the tests are generated, they have become deterministic - unit test code is checked in alongside the repository, CI (continuous integration) systems can prevent the code from merging to main branch until all CI tests pass. No LLM is involved in the test execution process, and if all LLMs stop functioning tomorrow, the tests still work just like they did before.
read anything an LLM wrote that’s longer than a hundered words? It’s usually immediately obvious
About 250,000 lines of code so far, and no… I don’t read it all - my personal music player isn’t that important, I just care that it works. Other more important projects I have read every single line. Even in the music player I have read a few hundred lines and skimmed a couple of thousand. Yes, the style is obvious. It’s what you might call “too complete” attempting to cover lots of edge cases that a human would usually neglect, though as human code is maintained over the long haul, it tends to mature in that direction. And just because it’s very complete doesn’t mean it’s perfectly correct, and that’s a major point of conflict with me and LLM code, I’m constantly asking it to simplify because the complexity is easy to hide mistakes in. You know what happens when you ask it to simplify? It simplifies.
people will also do reviews with an LLM and this sort of stuff will just accumulate in the codebase with nobody to reject it.
That’s a management / maturity problem. As new as all this stuff is, it’s virtually inevitable there will be an adjustment process. Our company runs on processes and procedures, they average 50 pages each and there are thousands of them. I am given almost 100 of them to “train on” on a near annual basis - whenever they get revised. No way in hell can I do an accurate review of 5000 pages of intertwined procedures, not code, natural language work instructions. LLMs can. LLMs are. Simple questions to an LLM like: “review this procedure, is it internally self consistent” will find issues, real issues - usually minor - but undeniable when you read the review, the problem it describes is absolutely present in the document, page 14 says “X” and page 23 says “not X” and the context is basically identical - so which are we really supposed to do: “X” or “not X”? Well, that depends. Depends on what? Additional context and detail that the already too long 50 page procedure doesn’t address. We just did an internal audit which turned up thousands of these mostly little issues. Addressing them is going to be… interesting. We’ve been in business over 70 years, we know we’re not going to tackle it all in a single year, there will probably be a plan to work down the worst of the issues, then a second round for additional polish, of course corrective actions to ensure that we at least don’t make things worse along these metrics with future changes… I bet by 2035 we’ll have 90%+ of today’s real observations fully addressed.
LLMs literally kill people both directly and indirectly
As does nearly every activity of life, both in human society and the natural world.
Why the are you getting more text than you can physically review at your job? That sounds like a recipe for poor reviews. How are you sure LLMs are not missing anything if nobody actually has the time to read through all the text? Why does the text that gets to a developer contain logical mistakes in the first place? Back at my last job we had people assigned to collecting feature requests, talk them through with whoever wanted the feature implemented and produce proper specifications that developers would implement without wasting time on specifications containing garbage.
As does nearly every activity of life, both in human society and the natural world.
Yea, all people who drank water died. Not a very good argument. Most things don’t have a mental disorder named after them. If I leave a webpage with a calculator running unattended it won’t magically persuade a random visitor into commiting suicide. Webpages with LLM-powered chatbots are doing just that. You may choose to neglect it, but to me it sounds like adding lead to your gasoline. Yea, your car runs better. Are you sure it’s worth it, though?
Why the are you getting more text than you can physically review at your job?
Regulatory required documented training, old conservative company.
How are you sure LLMs are not missing anything if nobody actually has the time to read through all the text?
You’re not. Neither do the humans tasked with the job do a 100% thorough review. Side by side, the LLM catches more flaws than the humans do.
produce proper specifications that developers would implement without wasting time on specifications containing garbage.
Sounds nice. We’ve got ~500 employees at our “site” (which is now distributed across 3+ states with remote workers), of those, about 8 do software development, so developers end up doing virtually all of the translation of customer “needs” to specifications / requirements. We also have a marketing department that changes their mind about twice as fast as the product development / delivery cycle.
If I leave a webpage with a calculator running unattended it won’t magically persuade a random visitor into commiting suicide.
That’s a touchy subject. A friend of ours recently had her son die of drowing in his bathtub, he was epileptic, had a seizure and fell in the tub. There was no suicide note, it was ruled an accident. He used ChatGPT to write his own obituary, as well as a book explaining his life from his perspective - each seizure was “taking away a little of who he was,” and the seizures were getting more frequent, none of the treatments or medications slowed them. It’s pretty clear, to me, that he was going to commit suicide and/or die soon with or without ChatGPT. After he passed, his mother had a ton of grief / guilt about not being closer with him in his life, but then… he actively pushed her away, even violently. It’s doubtful he would have written or expressed what he did as completely without ChatGPT assistance - how much was him, how much was LLM hallucination? Does it really matter? He wrote the book, and he put it where she would find it.
The next time you go driving, count the oncoming cars. When you get to 100, realize this: statistically, one - maybe two - of the drivers of those oncoming 100 cars will, eventually, end their life by suicide. And those are just the deaths officially classified as suicide. Engagement with ChatGPT undoubtedly works both ways, some cases accelerating the process or even causing them to take their lives when they wouldn’t have, and other cases slowing or stopping the pursuit of suicide altogether. It’s going to be hard to say if Chatbots or other world events drive the statistical changes starting a couple of years ago. The data isn’t clear yet, but the concerns have certainly been raised, and I would assume the supportiveness of Chatbots should improve as a result in the future.
I’m sorry you are working in such conditions, I hope you get better management that actually understands how much data one person can process.
I’m not sure how your story is at all relevant to what I said.
I don’t have a driver’s license, I don’t live in the US, I don’t need one. The lead in gasoline example was just the first thing that came to mind that fit a “technology that is profitable and useful, but kills people” description. Please pay attention to the road while driving instead of perfoming mental math exercises.
I would also like to ask you to think about what exactly you are arguing for right now, as I’m starting to think that our understanding of the value of human life differs quite a bit.
Human life is more valuable than money, property… but suicide is a reality of the human condition. It’s not something that has been meaningfully improved with modern psychology or medications. It’s worse among the poor but significantly prevalent across the wealth spectrum. Anything that interacts with people is going to interact with suicidal people.
Next time you are in a train station, count 100 people as they pass you - same effect. If you’re in a major station, look out across the big hall full of people, estimate how many hundreds of people you are looking at, and take in the reality of what 1 in 100 actually means.
You can add deterministic testing to AI output though?
Indeed, my code has never been more rich in unit tests now that coding assistants make them so easy to add. If the coding assistant subsequently does something weird that changes program behaviour it knows immediately.
Me as well. A year ago there was significant trouble with those unit tests not adding much value but that has improved a lot over the past several model updates. The unit tests aren’t 100% as good as they could or even should be (neither are human generated unit tests that take 10x as long to add) - but in terms of time invested vs value derived, they’re excellent value for effort.
About 90% of the time for me now, in Opus 5.5.
What do the tests do?
They check to make sure that the code continues producing the expected results. For example if I had some code that added up numbers, I might have a unit test that calls that function with the parameters “2” and “2” and checks to make sure the result is “4”. If something changes that makes it start returning something else then a flag is immediately raised.
Are these tests written by AI? Have you reviewed the tests? Do they cover all code paths and all the possible inputs and outputs? Do the tests have requirements cited (traceability)?
Unit tests have been around for a long time. They tended to be under-utilized in the past because they’re such a maintenance headache - changes / improvements in the production code that used to be a simple 5 minute thing can easily balloon to an hour or more of headache in necessary adjustments to perfectly good unit tests for “the old way” that happened to be impacted by the change.
Unit tests are a good way of keeping under-experienced programmers from accidentally breaking old things as they work on new things. They are VERY appropriate tools for LLM agent assisted development.
Absolutely not. In a world where hallucinating machines are writing code, humans should be writing the tests at a bare minimum.
AI writing code to pass tests written by AI will get people killed.
Boeing managed to kill plenty of people without a single bit of help from AI / LLMs in their MAX 10 software.
Used properly, LLMs could have articulated why their approach would lead to multiple crashes.
Used improperly, any tool is dangerous, from a chainsaw to a plastic spoon, the potential for injuries and deaths are unlimited.
Boeing’s executives had motivated reasoning to skimp on actual engineering and give the results the exec wanted over the objections of actual engineers.
Sound familiar?
Frankly, no. LLMs are so sychophantic, there’s little evidence they would have flagged a warning that what he executives wanted was dangerous.
And no, LLMs are not just some tool. Never in human history has a hammer given it’s users psychosis.
This is a completely nonsense argument.
The max issue on the Boeing planes wasn’t a software issue in that sense. The software actually worked as intended. Boeing just hid the fact that the software existed and made it purposefully hard to override, giving pilots quite the surprise when the plane didn’t operate the way they expected, especially when the angle of attack sensor wasn’t working.
Wouldn’t it be just… writing code yourself? It’s just instead of writing what you want directly you are writing tests for it.
But you really should be writing those tests regardless of whether or not you or an LLM ends up writing the actual code. That helps ensure that the code actually does what you think it does, and prevents bugs or unintended changes in behavior from sneaking in later when something changes
Well, depends. My thousand-line pet project does not need any tests, it needs to be feature complete and never touched again.
Bigger projects are usually ones where multiple people are contributing and having code that practically nobody wrote there is a huge problem on it’s own in my opinion, even if it passes a number of tests. Good tests themselves are usually harder to write than the code they are testing, I am not sure if delegating the actual fun part of development to an LLM and doing “chores” manually is a good idea.
LLMs write pretty good requirements, specifications, code, and automated tests from chat sessions. You do need to keep an eye on what it’s writing to make sure it’s not hallucinating, but hallucinations have been reducing in practice rather dramatically across the last 12-18 months of model development. They still happen, you will always need to guard against them, but usually a quick read of what is written followed by a quick test of the product will find the worst offenders. In the end, you need to check the final product to be sure it does EXACTLY what you want, just like you always did. And, it’s still a good idea (and as possible as ever) to keep the product modularized and also check every module to ensure it works as you want. How small you make those modules has always been a matter of preference / dictated by levels of concern, etc.
If you are generating tests with an LLM, it’s not deterministic.
Also, have you actually read anything an LLM wrote that’s longer than a hundered words? It’s usually immediately obvious. In the open source world things I keep seeing are exchanges like this one. And Niri is a fairly good case with it’s maintainer actually reading the PRs and catching all the bullshit that an LLM spits out. Very often nowadays people will also do reviews with an LLM and this sort of stuff will just accumulate in the codebase with nobody to reject it.
But even if it was as useful as big tech paints it to be, I’ll always be against using it based purely on morals. There is an enormous number of issues in that regard, most importatnly LLMs literally kill people both directly and indirectly. Condoning the adoption of this technology is either being ignorant or misanthropic.
Once the tests are generated, they have become deterministic - unit test code is checked in alongside the repository, CI (continuous integration) systems can prevent the code from merging to main branch until all CI tests pass. No LLM is involved in the test execution process, and if all LLMs stop functioning tomorrow, the tests still work just like they did before.
About 250,000 lines of code so far, and no… I don’t read it all - my personal music player isn’t that important, I just care that it works. Other more important projects I have read every single line. Even in the music player I have read a few hundred lines and skimmed a couple of thousand. Yes, the style is obvious. It’s what you might call “too complete” attempting to cover lots of edge cases that a human would usually neglect, though as human code is maintained over the long haul, it tends to mature in that direction. And just because it’s very complete doesn’t mean it’s perfectly correct, and that’s a major point of conflict with me and LLM code, I’m constantly asking it to simplify because the complexity is easy to hide mistakes in. You know what happens when you ask it to simplify? It simplifies.
That’s a management / maturity problem. As new as all this stuff is, it’s virtually inevitable there will be an adjustment process. Our company runs on processes and procedures, they average 50 pages each and there are thousands of them. I am given almost 100 of them to “train on” on a near annual basis - whenever they get revised. No way in hell can I do an accurate review of 5000 pages of intertwined procedures, not code, natural language work instructions. LLMs can. LLMs are. Simple questions to an LLM like: “review this procedure, is it internally self consistent” will find issues, real issues - usually minor - but undeniable when you read the review, the problem it describes is absolutely present in the document, page 14 says “X” and page 23 says “not X” and the context is basically identical - so which are we really supposed to do: “X” or “not X”? Well, that depends. Depends on what? Additional context and detail that the already too long 50 page procedure doesn’t address. We just did an internal audit which turned up thousands of these mostly little issues. Addressing them is going to be… interesting. We’ve been in business over 70 years, we know we’re not going to tackle it all in a single year, there will probably be a plan to work down the worst of the issues, then a second round for additional polish, of course corrective actions to ensure that we at least don’t make things worse along these metrics with future changes… I bet by 2035 we’ll have 90%+ of today’s real observations fully addressed.
As does nearly every activity of life, both in human society and the natural world.
Why the are you getting more text than you can physically review at your job? That sounds like a recipe for poor reviews. How are you sure LLMs are not missing anything if nobody actually has the time to read through all the text? Why does the text that gets to a developer contain logical mistakes in the first place? Back at my last job we had people assigned to collecting feature requests, talk them through with whoever wanted the feature implemented and produce proper specifications that developers would implement without wasting time on specifications containing garbage.
Yea, all people who drank water died. Not a very good argument. Most things don’t have a mental disorder named after them. If I leave a webpage with a calculator running unattended it won’t magically persuade a random visitor into commiting suicide. Webpages with LLM-powered chatbots are doing just that. You may choose to neglect it, but to me it sounds like adding lead to your gasoline. Yea, your car runs better. Are you sure it’s worth it, though?
Regulatory required documented training, old conservative company.
You’re not. Neither do the humans tasked with the job do a 100% thorough review. Side by side, the LLM catches more flaws than the humans do.
Sounds nice. We’ve got ~500 employees at our “site” (which is now distributed across 3+ states with remote workers), of those, about 8 do software development, so developers end up doing virtually all of the translation of customer “needs” to specifications / requirements. We also have a marketing department that changes their mind about twice as fast as the product development / delivery cycle.
That’s a touchy subject. A friend of ours recently had her son die of drowing in his bathtub, he was epileptic, had a seizure and fell in the tub. There was no suicide note, it was ruled an accident. He used ChatGPT to write his own obituary, as well as a book explaining his life from his perspective - each seizure was “taking away a little of who he was,” and the seizures were getting more frequent, none of the treatments or medications slowed them. It’s pretty clear, to me, that he was going to commit suicide and/or die soon with or without ChatGPT. After he passed, his mother had a ton of grief / guilt about not being closer with him in his life, but then… he actively pushed her away, even violently. It’s doubtful he would have written or expressed what he did as completely without ChatGPT assistance - how much was him, how much was LLM hallucination? Does it really matter? He wrote the book, and he put it where she would find it.
The next time you go driving, count the oncoming cars. When you get to 100, realize this: statistically, one - maybe two - of the drivers of those oncoming 100 cars will, eventually, end their life by suicide. And those are just the deaths officially classified as suicide. Engagement with ChatGPT undoubtedly works both ways, some cases accelerating the process or even causing them to take their lives when they wouldn’t have, and other cases slowing or stopping the pursuit of suicide altogether. It’s going to be hard to say if Chatbots or other world events drive the statistical changes starting a couple of years ago. The data isn’t clear yet, but the concerns have certainly been raised, and I would assume the supportiveness of Chatbots should improve as a result in the future.
I’m sorry you are working in such conditions, I hope you get better management that actually understands how much data one person can process.
I’m not sure how your story is at all relevant to what I said.
I don’t have a driver’s license, I don’t live in the US, I don’t need one. The lead in gasoline example was just the first thing that came to mind that fit a “technology that is profitable and useful, but kills people” description. Please pay attention to the road while driving instead of perfoming mental math exercises.
I would also like to ask you to think about what exactly you are arguing for right now, as I’m starting to think that our understanding of the value of human life differs quite a bit.
Human life is more valuable than money, property… but suicide is a reality of the human condition. It’s not something that has been meaningfully improved with modern psychology or medications. It’s worse among the poor but significantly prevalent across the wealth spectrum. Anything that interacts with people is going to interact with suicidal people.
Next time you are in a train station, count 100 people as they pass you - same effect. If you’re in a major station, look out across the big hall full of people, estimate how many hundreds of people you are looking at, and take in the reality of what 1 in 100 actually means.