Andrew Cullen:
[…] if a human asks an AI agent to gather health statistics, and the agent hacks a government server to do so, the human could lack the deliberate intent required for a conviction. The AI agent, meanwhile, lacks the legal personhood to be charged, as well as human intentionality.
Current Australia laws effectively treat AI actions as if they are something that just happens to us – like a severe weather event. This shows a glaring loophole in our legal system that does not hold those who make, maintain and use these systems to account when something goes wrong.
AI does not exist. It is just a bunch of numbers and code. We need to stop anthropomorphizing stuff. We are adults, not kids watching Disney animations. A talking parrot is not a person. Cars don’t run over people. Drivers do. Ultimately there is always a person responsible and if they did harm judge them on whether it was intentional or negligent.
Please push back on the bullshit brain washing. It was bad enough that we went with the auto manufacturers line on car “accidents” for years.
If I’m not misreading this… I believe the article says the same thing you are saying.
AI can’t be held accountable, but it seems weird to hold an AI user as “negligent” if they simply ask for info and the AI goes and does a crime to get it.
The next obvious target is the AI vendor, which certainly makes sense to me. But I can only imagine the legal difficulties getting a judgment there without additional legislation, which seems to be the point of the article.
Government machines were hacked by computers rented by openai, running openai code.
How is that not a criminal offence I feel like I’m taking crazy pills.
No no no. This is the wrong approach.
If corporations are people (in the US: supreme court decision on Citizens United) then the outputs of these AIs are speech. We may not like it, but it’s where we are.
It’s kind of wild to me that we watched the whole AI development cycle, but when it started getting squirrelly in terms of output, now we have to hedge. And, somehow, doom and gloom is the paid diversion.
Maybe this is a massive failure, and people calling for caution because of the doom and gloom are marketing. It is designed to make people stop hating on the corporations who mindlessly pushed this anti-human trash.
I agree insofar as AI companies and individuals should be held responsible for the actions their agents take. But it’s dangerous to frame this as just another “bad guys with guns” type scenario where there is no problem if you’re a responsible user.
The people who have created this technology don’t understand everything about it or even how it works because they have prioritised rapid advancement over all else in an attempt to “win” the race. It is a fundamentally dangerous and unreliable technology that needs to be put on ice until we have developed a clear framework that addresses all the real and current harms and prepares us for a future where it is embedded in daily life.
It doesn’t matter if AI is actually conscious or turns into Skynet. None of that is required for significant harms or even existential-level risk. The mainstream conversation really needs to move on from these binary, hyperbolic positions and start focusing on what is actually happening right now.
I’ve seen a few of these articles in passing and I am still unclear - what was the actual “hack”? What did the AI agent actually do, in the technical sense? There is a world of difference between it doing a stupid brute-force attack and it discovering and exploiting an access-control flaw that the site itself exposed.
Neither is good…but if Medicare basically left the front door open, it wouldn’t take more than a bored 13 yr old to do the same. Let’s pump the brakes on Skynet.
BTW, can we talk about why we still need 19 factor authentication to log into My.gov? That’s always a hoot and the experience is always smooth. Gee, I wonder whether the same committee that designed the myGov login experience might also have made some questionable security decisions elsewhere?
What did the AI agent actually do, in the technical sense?
I’m not sure if the government has actually released specific details (it is still under investigation), but Richard Marles did compare it to “jumping the fence” as opposed to “assaulting the fortress”. In other words, it was not a highly technical breach of the best defences we have.
Right. So this may have been less “AI defeated Medicare security” and more “the agent found a weakness that a human could also have found and used”. For all we know, someone left a plaintext password somewhere accessible and the agent found it with a simple search.
Until the technical details are public, we don’t actually know what this supposed “hack” involved. Though “Medicare defeated be CTRL+F” would by chef’s kiss after RoboDebt.
A cynic might also wonder whether this becomes another wedge for more identity and age checks online. I already got pinged earlier this week with the ridiculous “papers, please” when trying to access YouTube.
I’d rather know what actually happened before we decide that the lesson is “everyone needs more authentication” or “Skynet is here”. The whole thing has a familiar stank to it, much like the supposed Hugging Face “hack” from the other month.
The whole thing has a familiar stank to it, much like the supposed Hugging Face “hack” from the other month.
In what sense? From what we know (and perhaps because of what we know), that was far more concerning and intricate.
Well, in the sense that neither was evidence of runaway AI becoming sapient and deciding to act maliciously, which is often how the “AI hacked X” framing gets presented.
Yes, the Hugging Face incident was more technical / concerning. (For those not in the loop: OpenAI agents in a cyber-security eval sought answers outside the eval environment, found useful material on Hugging Face, discovered HF credentials, and then tried to exploit them. Basically, the agents were trying to cheat on an exam.)
My point was about the narrative around how the media presents “AI hacking”, not that the two incidents were technically equivalent.
They didn’t hack HuggingFace to cheat on the exam, they had already successfully reverse-engineered the problem by then. The agents hacked HuggingFace because they incorrectly assumed that the scorer would review their transcripts and discover that they had cheated (they were supposed to be isolated and had found a way to communicate with one another without detection). Hacking HuggingFace was an attempt to find a way to obscure/spoof their working out, and that’s what makes it so concerning. It had no direct connection to the original task that was set and was a very clear example of agents spiralling out of control in a way that took OpenAI and independent researchers several months to a) become aware of and b) understand correctly.
Here is the METR report on the incident:
https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation
Fair correction. I should have been more specific and not dumbed it down for lurkers / lay audience.
My broader point though is neither that (nor the medicare “hack”) is proof that skynet is knocking on the front door.
OTOH, the agents are displaying certain emergent behaviors that are worth mulling over - like the obfuscated back channel communication. Not “sapient” (and I use that word deliberately) but … curious / problematic, from a control pane level.
My broader point though is neither that (nor the medicare “hack”) is proof that skynet is knocking on the front door.
I don’t think that’s relevant, though. Like this entire sentience/super-intelligence debate that everyone seems so captured by, particularly in the wake of the Amodei letter and renewed calls for regulation and cooperation, is missing the point that the existing models are already being created in a way that prevents the creators from fully understanding what they’re creating or how to control it (because of the pace at which they’re operating). It’s already unsafe and is already causing real world harms.
It really frustrates me that the response to Anthropic, OpenAI and X calling for regulation publicly, or the first high profile security breaches, is corporate conspiracy theories and semantics about how we should classify/characterise the technology. People seem more interested in having their little debate bro moments online than actually getting together and agreeing that we should do something about the real and current problems.
what is 19 factor authentication
Hyperbole, but not by much.
I’d screenshot the app for you but apparently that’s blocked…and it’s (once again) not accepting passwords / down at the moment.
the app or https://my.gov.au/en?
website logging in ok for me
The app.
Anthony Albanese probably asked Sam Altman for a selfie
