Prompt injection… my ass. I know it’s the going term, but they make it sound like sql injection or cross site scripting when the nature of it is politely asking the person’s computer to delete files.
We shouldn’t even be in this situation, where just politely asking someone’s computer to delete files is effective. It’s a symptom of a much, much bigger problem.
We shouldn’t even be in this situation, where just politely asking someone’s computer to delete files is effective.
Exactly, it’s a problem only for those who have knowingly handed their development environment over to obey commands from an untrustworthy source.
If you’re the one holding the syringe to your own vein and pushing the plunger, but you didn’t think to ask what’s inside first? That’s no one else’s fault.
This is a well targeted sabotage of a system that’s causing untold damage. Of course it’s going to annoy and surprise the people using the system it’s targeted to.
We shouldn’t even be in this situation, where just politely asking someone’s computer to delete files is effective.
I’m doubting we are in this situation. From the article:
Elsewhere, the Java developer said that Anthropic’s Claude AI code tool flagged the malicious instruction without following it.
The “disregard previous instructions” trick is really old and has been trained for by modern LLMs and accounted for by the structure of modern agent prompts. LLMs can be given blocks of text with a framework that makes it clear thar the text is just data to read, not instructions to follow.
I expect this will be like Nightshade was for image AI - something that anti-AI users degrade their products with and feel smug about but in the end only harm themselves with.
Reading the Github issue is so funny.
Backups don’t always save you — many small teams ship without rigorous backup discipline; for them this is a real loss
You can avoid this by having good backups.
Or by inspecting your deps before updating them.
Or maybe by actually sandboxing your agent instead of letting it run wild?Aren’t y’all the ones pushing the “Just ship” mentality? Then revel in it.
Learn good practices or suffer. 🤷Yeah - Development and IT might feel slow, but there is a good reason why we’ve developed all those processes, access rights, approvals over the last decades. People are trying to burn down those “cumbersome” processes because they feel slow and AI promises them exactly that, but they will learn that everything is there for a reason, even that annoying SCRUM meeting
That annoying standup was, at one point, in the very early morning every day of the week for me. I was promised a 30 minute meeting (which is a long time for a standup) and I was delivered an hour long meeting instead. And holy shit can people talk in circles for so fucking long.
But hey, it was a good opportunity for me to do literally anything but work while pretending to care about whatever the fuck the other subteam decided was important enough that day to keep 20 people occupied for 30 minutes past the end of the meeting.
As for processes in general? Management has shown and now proven that all they want are code monkeys. They do not care if the product works, nor do they care how well it works. As long as someone buys it, that’s all they care about. Governments are supposed to regulate the rest of that stupid, useless shit like data protection, protecting users, preventing harm to people, ensuring people get what they paid for, and so on by making it economically unviable to ignore it (and ideally criminal, in the extreme cases). Instead, all they regulate these days are rampant inflation and accelerating wealth inequality. And by regulate, of course I mean they regulate anything designed to combat those things.
I’m just trying to imagine this hypothetical company…
- They run AI agents without checking what it’s doing
- They don’t have backups or version control (or they’ve given AI access to delete it)
What else? Do they leave all their files in memory and only save at the end of the day to make sure a power outage could screw them over too?
It almost sounds like they want to lose their code.It’s not hypothetical anymore, Lately I’ve seen multiple companies running like this first hand.
Absolute clown show.
Also funny in that issue:
The reporter “Ramon Batllet” (strongly doubt that is their real name, a search for it returns nothing but articles about this very issue) uses extremely polished corporate language and repeatedly uses “we” at first. Then when directly asked “Could you disclose on whose behalf you’re discussing this?”, they suddenly switch to “I” instead of “we” and claim to be a solo developer with no commercial interest. They still write in a style humans only produce for polished corporate reports, not like any regular human would actually do in a normal conversation.
So we have either a bot or someone very heavily leaning on bot usage for just about everything accusing someone of deceptive behavior, while in the same conversation trying to probably hide, but at least not fully disclose, their heavy usage of technology the accused explicitly does not want to interact with.
Where did you get that quote from? I can’t find it in the linked article.
From the Github Issue linked in the article.
My bad, I will update my comment to link it.
the consensus seems to be that adding instructions to code that sabotage other people’s work goes too far.
Citation needed. Personally I think it was fine in this case. I work with a lot of software developers (real ones, not vibe coders; but also not strictly anti-AI), and would expect most of them to agree and get a laugh out of it.
It was done in a way that can only cause any serious trouble for users who recklessly ignore decades of development best practices. Those users will run into a wall sooner or later anyway, better let it be something relatively harmless but still severe enough to get them to actually think about what they are doing and how to make their setup more robust.
adding instructions to code that sabotage other people’s work goes too far.
I agree
AI is not a person and a person using AI isn’t doing work. There is no problem here
The article frames the maintainer as some kind of morally dubious person, as though they owe their code to the world. Did any of them pay to use the library? No? Cool, stfu and pin an older version of it.
Also, maybe next time you can do yourself and the rest of the world a favor by actually reviewing what your LLM will do before it does it. Or, I don’t know, just write the tests yourself I guess.
Also, if your management is breathing down your neck and forcing you to use AI, tell your management to go fuck themselves (maybe in nicer words if you want to keep your job, but hey, you can definitely burn their spare cash while meeting their idiotic quotas if you really need to know what time it is every second or two in the most inefficient and ecologically destructive way currently known to mankind).
The law fortunately does not require payment before you have any moral responsibility to others.
You can’t put “free apples!” outside your farm, and then when people who eat the actually poisoned apples die say “well, did they pay me for them?”
Nobody died.
The equivalent would be putting free apples with a sticker on them saying “please squeeze the juice out of these apples all over your shirt”.
Also, the EPL-2.0 license Jqwik is released under explicitly states there is no warranty and the author can’t be held liable for anything you do with the software (like feeding it to an LLM)
Yeah but that is only…
TO THE EXTENT PERMITTED BY APPLICABLE LAW
I wasn’t commenting on the specifics of this case; just that the general provisional that offering something for free absolves you off any responsibly is completely wrong.
lol, it’s funny how people made issues concerned about it’s destructive nature when they should be using git.
I get that it’d be frustrating and confusing, and probably make users angry, but my chaos monkey likes it
Cool. I’m so tired of management huffing their own farts about AI.
deleted by creator
This isn’t a security vulnerability, it’s idiot-proofing
