Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid - welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.
The post Xitter web has spawned so many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.
long rant about math
The recent big AI results in math have left me in quite a bad mood. I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs. Back in the days of pure scaling LLMs and Sam Altman talking about Dyson spheres, I was skeptical that LLMs would do math, but I did think that perhaps in the future, techniques using these formal languages could contribute to math. Well, it seems like OpenAI and Anthropic had the same idea and I underestimated their limitless checkbooks. Many of the biggest results were announced by mathematicians directly working for them (and presumably being paid a handsome amount).
For what it’s worth, after the last of these big announcements, I decided to try one of these AIs on one of my small problems that I couldn’t figure out. The AI did give a solution. That is, until I checked it thoroughly and realized that the it had a subtle but severe mistake that made it useless. I reprompted it, it failed again, and I ran out of tokens. I’m sure someone will tell me to shell out $200/mo for a pro subscription.
In the math and computer science research community, this is all anyone can really talk about right now. Honestly, after watching this whole AI bubble starting from the very beginning, I think the AI companies want to use marketing to stoke fear that all mathematicians will be replaced. But now, I am just too tired to argue. The amount of alarm and the extraordinary social pressure to use LLMs has soured me to this whole research thing. If becoming a researcher will one day require supporting these evil AI companies, I would rather just not. My dream job now is Factorio developer.
A lot of annoying people in technical areas view the world in terms of an intelligence hierarchy: the smartest people do math and physics, the slightly less smart people do coding, and the dumb people do everything else. So if AI can do math then it can do anything else. But, as an example, it is abundantly obvious now that AI is not replacing filmmaking. The techbros might be moved by arguments about how hilariously expensive video generation is, and how all these videos are 2 second clips stitched together so you won’t feel the uncanny valley. But the real reason is that nobody wants to watch slop made with no intention or feeling. Also, nobody wants to support the AI companies, which could not act more evil even if they tried.
The mania in math right now quite resembles the mania in software engineering back in December-February, when Claude Code definitely solved all coding. I don’t think the boosters expected that by April, everyone would be complaining about how expensive it all was while seeing an endless parade of vibe coding disasters (and no increase in productivity). Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills. They would need to find a use case in the real world, where hallucinations can cause serious damage and cannot be formally prevented. And they have certainly tried. Math will not change the fact that all of this will collapse.
I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs.
100% this. Also, looking back at an earlier example that was actually written up in more detail, AlphaGeometry 1 got 28/30 problems, but entirely stripping out the LLM from the system, the symbolic logic proportion alone could get 14/30, and replacing the LLM with different heuristic methods could get 18/30 and 21/30 (for different methods).
Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills. They would need to find a use case in the real world, where hallucinations can cause serious damage and cannot be formally prevented. And they have certainly tried. Math will not change the fact that all of this will collapse.
The boosters and LLM companies still believe LLMs get their current level of performance by generalizing and not just memorizing facts (and maybe a wide shallow pool of weak heuristics). So they are hoping by pushing the LLM performance up in some narrow domain they can churn out synthetic data for, they will see some large general improvements in LLM performance.
Am I right in understanding that almost all the big name results in LLM-derived math recently come from big publicity projects in which someone spent ungodly amounts of money to have the thing nondeterministically fuzz huge numbers random seeds leading to independent random outputs around a topic, putting out simulacra of ideas which could be then deterministically algorithmically checked? In fields where something like finding one counterexample to a conjecture would be a big deal, or where you just need to try a huge number of possible solutions until you happen to hit on one that works, rather than follow a long train of logic?
Among the three big results I’ve looked at (unit distance problem, cycle double cover, Jacobian), two were counterexamples and one of them had a short 3 page proof using ideas from the 1970s. The Jacobian conjecture is an extreme case because a single counterexample is enough (for unit distance, you technically need a family of counterexamples), and it is easy to check with very basic computations. It is telling that all of these announcements came from OpenAI or Anthropic employees, who presumably have unlimited access to their AI. Nobody really knows how many resources they spent on this, or what else they tried. Nobody really seems to care about this question, either.
I think there is a phenomenon where supposedly hard questions are much easier than expected, because by chance nobody found the right approach for a while, and eventually it becomes famous as a “hard problem” which makes nobody want to attempt it.
What I’m more worried about is many people starting to use AI to try and prove small lemmas for them in their projects. Of course, a $200/mo subscription is absolutely necessary to them. This honestly feels like a repeat of Claude Code back in February. The software engineers eventually realized that AI is absurdly expensive after the AI companies realized that spending $14000/mo to service a $200/mo subscription is a bad idea. If the AI vendors couldn’t squeeze money out of rich software companies, what exactly are they gonna get out of poor mathematicians and universities? Also, there is the cognitive decline caused by overuse of LLMs that has yet to set in.
A lot of annoying people in technical areas view the world in terms of an intelligence hierarchy: the smartest people do math and physics, the slightly less smart people do coding, and the dumb people do everything else. So if AI can do math then it can do anything else. But, as an example, it is abundantly obvious now that AI is not replacing filmmaking.
Going by those annoying peoples’ logic, filmmakers are smarter than coders, because LLMs can (allegedly) program, but they can’t make a good film. I have no wider point to this, I just find this really, really funny
“The mania in math right now quite resembles the mania in software engineering back in December-February, when Claude Code definitely solved all coding. I don’t think the boosters expected that by April, everyone would be complaining about how expensive it all was while seeing an endless parade of vibe coding disasters (and no increase in productivity). Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills.”
^MBAs at Open AI desperately trying to figure out who is willing to buy a counter example for 100 billion USD . pee en gee
I reprompted it, it failed again, and I ran out of tokens. I’m sure someone will tell me to shell out $200/mo for a pro subscription.
One of the things that’s never clear from the reporting on ai successes is exactly how much actual paid human time went in to achieving those successes. This was especially notable in the fable-based security work… a huge amount of person-hours went into turning fable-detections into actual meaningful vuln reports.
A lot of demonstrably clever and capable people are involved with the llms-for-maths work, and a lot of money was spent on their time and supporting their work. Replicating it without your own stable of mathematicians and computer scientists and all the tokens they can eat is probably impractical.
I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs.
Fwiw, lean is a general purpose programming language, though despite microsoft’s efforts no-one uses it for that. I think its popularity with mathematicians came as a bit of a surprise.
Anyway, the other important thing that didn’t get reported on is that building the formal definition of the problem is not trivial! Obviously I don’t need to tell you that, but from the reporting you’d think that an llm solved all these problems, when in fact it was an llm in the hands of some very capable people who absolutely did not just prompt the thing in plain english.
Anyone hoping for self-marking homework here is going to be disappointed… lean slop confirming to formal spec slop is just expensive slop. Reviewing regular genai code is awful, even the thought of reviewing genai dependently-typed code makes me want a new career.
It is surprising how many exceptionally strong mathematicians have started working for OpenAI and Anthropic. These people would have easily become professors at top universities if they stayed in academia. I think many mathematicians, especially the competitive ones at the top, have a “progress at any cost” attitude (and I’m sure the paychecks helped). As for the results, you still need good mathematicians to sift through all the output to identify that the proofs are valid.
I would honestly be positive about universities developing their own specialized math AI (in an ethical manner) to help mathematicians get these kinds of results, but right now, AI is inseparable from these evil companies. Thankfully, I believe this is a likely outcome in the future because the AI companies will one day implode.
From what I’ve seen, most prompts are in plain English. I suppose the part where the AI parses the statement correctly is much easier than the part where it boils a couple lakes in the process of bashing its head against the wall trying millions of different combinations of random shit from the literature to slap together a proof. For one of the big results (cycle double cover), the prompt specified that the AI could use 64 subagents and was required to not give up for at least 8 hours. The tokenmaxxers would be proud, we didn’t need that forest anyway. Thank god math doesn’t have a CTO to look at the expense reports.
deleted by creator
I just entered university for math and even though this is all very demotivating, it’s just what I’m good at.
https://math.andrej.com/2013/08/19/how-to-review-formalized-mathematics/
The AI people’s cry of "no don’t look at the code! it’s in lean so it’s correct! does give me a bit of hope (hi bitofhope if you’re here) that it’s bullshit that will fall over
I think a serious possibility is that AI generated papers flood the zone with uninteresting incremental results that are eventually meaningless and full of mistakes. Right now, math is full of smart, dedicated people, so at least major results are reviewed carefully. But as AI alarmism drives away many honest people from the field, the remaining mathematicians will be burdened with far more work to review, and their cognitive faculties will be eroded by LLM use. Despite 4 years of development, $3 trillion of debt, mountains of stolen data, all the agents and harnesses and loops and other expensive tricks, as well as the advantages of Lean in math research, LLMs still hallucinate.
I believe this is happening with software, but at least there are objective consequences for screwing up there (guy gets his home directory deleted, email is sent on a guy’s behalf without permission, small business gets every customer subscription cancelled). But nothing bad happens if there is a mathematical mistake in a paper and nobody catches it. One could say to just provide a Lean proof, but there is still the issue of making sure the Lean code actually matches the content of the paper. Exactly what force will correct things?
Still, I don’t think this is the most likely possibility. The AI companies are extremely unsustainable financially, and it’s not like they’re very popular. Once they collapse, I believe there will be a re-evaluation of how LLMs should be used in research. If they are used (let alone trained), someone is going to have to pay the bills.
In the end, we have to ask ourselves the question of why one does math. To me, math is not really a field where you memorize trivia. The real value comes from being able to think abstractly and rigorously from first principles, and from understanding why something is true rather than just knowing it is true. It is another aspect of your ability to reason as a free human. A few dedicated people go into math research, but your skills can easily go to many places. If you’re starting undergrad, you have plenty of time to see how this all pans out before making a decision.
Biologist here.
This REALLY reminds me of how jealously cells guard their genomic DNA from interaction with nucleic acids out in the environment.
Most genetic information on Earth is malicious information, selfish replicators in the form of viruses or transposable elements or selfish elements. Things that subvert the signals within a cell for their own propagation and provide nothing productive that the cells care about. So cells jealously guard their own genomic DNA and have all kinds of checks to make sure that nothing other than that sequence gets used, and outside sequence does not get incorporated into it. ANY DNA in your cytplasm gets rapidly destroyed, double stranded RNA sets off your immune system like crazy, even RNA with sequence statistics that are not quite like that of your species can set off an inflammatory reaction, immune system cells seeing RNA inside them that is overly compact and optimized like viral RNA treat them as sources of antigen rather than self.
I cannot help but think we are living through the transformation of our non-brain-information sphere into a state like that of the genetic information sphere. Most material out there being meaningless for our purposes and us needing to jealously guard the provenance of information we use so as to not use bull, or worse, huge amounts of malicious information made to subvert us to the purposes of the powers that be that generate it.
Evolution makes parasites more reliably than anything else. How did we train text-generation systems? Basically, to mimic the written word on the page like a stick bug on a stick. They’re like those beetles that live in ant colonies, sending out social signals that make the ants see them as offspring that have to be babied rather than parasites that don’t contribute. They replicate the form while not being the thing that they have subverted the signals of being.
EDIT: There is something wrong with the upvote counter
> even RNA with sequence statistics that are not quite like that of your species can set off an inflammatory reaction, immune system cells seeing RNA inside them that is overly compact and optimized like viral RNA treat them as sources of antigen rather than self
I was aware of the other DNA/RNA recognition/defense mechanisms, but not of the ones I quote from your toot, here.
May I kindly ask for some references/sources? I’m quite interested!
I kind of read wayyyyy too many preprints and some of that is the result of super briefly summarizing some things I have read recently. Here:
https://www.biorxiv.org/content/10.1101/2024.11.26.625518v2 poor codon optimality for your translation system leads to immunogenicity and activation of innate immune signaling in animal cells
As for length and super optimized proteins, it’s mostly about RIG proteins (see https://www.pnas.org/doi/10.1073/pnas.1005077107 for an old bit of a review) and the whole DRIP hypothesis about how short mismanufactured proteins are preferentially the source of presented antigens
@BioMan @lagrangeinterpolator There are a few things we have forgotten as a species. Our forgetting will prove disastrous.
-
The acquisition of knowledge is a *social* process. Truth does not exist is a vacuum. It is the outcome of social processes.
-
Our default mental and social processes do not automatically produce objective truth. Far from it, in fact. Our default is mob consensus.
-
Our current success rests upon the advancements of The Enlightenment, which developed social processes (like the Scientific Method) which tend, over the long run, to create local knowledge that approaches objective truth.
-
@BioMan @lagrangeinterpolator I love this explanation!
you know, I’ve been thinking thoughts along this line (starting from my earlier comments on awful of the creeping loss of the open internet to gated communities) progressing along the current arc of “with teeth” stuff necessary in today’s infosphere, and this is a hugely helpful perspective!
it’s pretty weird seeing a coming time when the old mostly open-and-connected-widely internet is … not. even now it’s already on us quite a bit (geoblocks, countries doing specific shutdowns, whole areas being handled with less trust, etc). it all follows cultural lines, of course, and I often wonder whether that will see some kind of directed effort to change (for the better) after some of the coming dark years
Part of me thinks its an inevitable result of any information system in which sending a message requires a small enough amount of effort.
@lagrangeinterpolator @flaviat This is going to be more of a problem in the humanities than the sciences because in the latter we know there is a right and a wrong answer without which things don’t work. In the humanities there is no right answer to check against.
The zone has been flooded with crap since before LLMs even arrived because of publish or perish.
Ah, I’ve hit an extremely satisfying professional milestone and thought I’d share:
“Being acquainted-enough with the codebase to be able to, with full conviction and a lot of glee, answer my LLM-head colleague simply: ‘Claude is wrong’”.
(Claude was extremely obviously wrong once you spent more than half a second thinking about its claim.)
Found a remarkable ad-hoc anti-LLM-product manifesto on the orange site. As is tradition, tag yourselves; I’m “believ[ing that] there’s a consistent pattern of cognitive decline in the pro-LLM folks.”
my god, you found the good HN comment
Ooh. I love these. I’m “I don’t think Sam Altman or most of his social circle are good people and I don’t want to give them more wealth and power.”
I’ll go one step wider. It is not just Sam Altman or his social circle, but all the people most enriched by capitalism (and thus most able to exploit and benefit from a technology that shifts power away from labor and to capital) that I don’t want to see further empowered.
I hate LLMs because I am under enormous pressure, from many different angles, to also use them. I’m being asked – required, in some cases – to ignore all of my concerns about them and use them anyway because everyone else does and “obviously” they are the future. Because, somehow, all of my positions against LLMs make me unreasonable and a less valuable member of society.
oof. yeah that’s me.
It’s as if, practically overnight, everyone went from, “we need to do everything we can to reduce our demands on the environment or we’re cooked” to “oooh, shiny new toy, and it only ruins everything around it, cool!”
Definitely this one for me. I can only describe it as some fucked up denial psychology, the way people are pretending the environment isn’t collapsing, and the LLM boom on top of that is like a bad joke. Even in discussions critical of the tech sector this is treated as almost an afterthought—the very real, very violent damage done to all of us from poor countries by people in rich countries to prop up their toy fantasy schemes. Even if LLMs did everything they claim to do, it would still be a moral imperative to form insurgent guerillas to blow up these datacentres, on the environmental damage they do alone which is not a minor detail Karen it’s kind of a big deal
How can we, as a community, talk about the existential risk of climate change or the benefits of adopting renewable energy or cheer the development of ever-more-efficient power consumption
Well I have a good/bad news thing about that one
Could be sort of an age related thing then, no?
and also crypto https://codeberg.org/Codeberg/org/pulls/1254 surviving cryptobros are nonplussed
there’s a new variant of the old crypto affinity grift around, too: banning crypto projects is transphobia.
ah yes famously non-far-right cryptobros and downright woke russian and iranian sanction evaders
Their “Dev team of none” justification (one of many) is really solid and not something I’ve seen spotlighted before, here it is in full:
The development team of none
Using LLMs to work with your code gives you a kick of adrenaline. You can develop at a rapid pace, build things as if you had a large team. Only that you have none. In fact, you are (often) alone, working with a statistical machine that turns energy into code.
It seems like many ‘vibe coders’ don’t realize that they don’t actually have a community around them. They build projects as if they had, and spend resources accordingly. We see projects having a lot of code activity, heavy CI/CD testing, frequent and large release binaries. Sometimes, it feels like the amount of supported platforms exceeds the amount of actual users.
To us, it seems ridiculous to see projects with a single developer and virtually no users consuming as much or even more resources than some of the largest community projects on Codeberg, which operate frugal with CI/CD and storage resources. We do not believe it is reasonable for Codeberg to invest our precious donation money into hosting of large ghost projects.
Daft ai project of the week: “lore”, a version control system that should not be confused with another vcs of the same name open-sourced by epic games a few weeks ago.
lore does not track your code. it tracks the prompts, notes and decisions that produced it. you commit intent. when you want code, you run lore materialize: it replays the accumulated intent into a brief, and an agent reconciles the working tree to match it.
the code is build output. the intent is the source.
…which would be great, if the output of llms were deterministic, and generally it is not. This means rolling back a change to undo or fix something is just another spin of the roulette wheel, with the analogy being reinforced by the number of tokens you’ll have to buy and burn to do a rebuild. It looks like it still needs a real vcs behind the scenes, given that it isn’t entirely self-hosting, but maybe I misunderstood something.
You might wonder if this is some kind of satirical work, or perhaps piece of performance art, but I’m fairly certain it isn’t… the creator and sole human author (naturally, claude is the only other “contributor”) is a big fan of something called “open audio protocol” and a project called “audius” which came about by someone asking what if spotify and soundcloud, but on the blockchain? With ai agentic integration?
I’ve always thought that the ability for a VCS to show me exactly what my codebase looked like at any point in the past wasn’t as dementia-simulating as I’d like
Finally a VCS that simulates the horrific results of
cvs mergegone wrong, 20 years since the last time I used cvs.Wait, did I get that right and it reruns the prompts every time you want to look at the code? Holy token costs batman.
Perfect for when your workplace will fire you if you dont use enough tokens, I’m sure.
kayfabe vcs
Considering this is the logical endpoint of the “LLM’s are just like compilers bro” line of thought, I’m not surprised at all.
Nah, this isn’t the endpoint… you want something like prompt2exe, which is a performance art piece. Gets an llm to output machine code, and bundles it into an executable.
no package dependencies and invokes no assembler, compiler, or system linker.
Hardcore mode. No handholding for puny human devs by their tool chains. 🦾
Probably impractical for normal people to run, unless they have an employer who wants their minions to be burning all the tokens and pays up for the expensive kind of accounts.
Jesus, you’re right. This is so much worse.
I think there’s something to the compilers analogy, in the sense that if you used a C compiler through a chat interface with one liners to insert snippets into assembly without keeping the source code in source files as such, it would probably be slower than just writing in assembly yourself.
That LLMs have a number of extremely undesirable properties as “compilers” (from nondeterminism to plagiarism) does not make the dumb ass chat interface any less fundamentally shit. It just makes them shit in other ways too.
There is one narrow case in which I’ve found the compiler analogy useful: explaining my frustrations with LLM-assisted design/architecture doc generation. Here, it’s useful because handing someone a design document that was output by an LLM is a lot like handing someone a binary and asking them for a code review.
I’d much rather work with whatever disorganized soup of thoughts someone fed into the machine, because those at least contain intent, rather than a document that actively obscures what the designer wanted in a barrage of detail.
Yeah… I think the core point is, LLM’s shit is generated artifacts, which need to be kept separate from sources. That you can’t actually re-generate the same artifacts, is just a dingleberry on top of the shit pile.
I know we’re all familiar with the metaphor of LLMs as roulette wheel, but I think there’s some real craft to turning that into Russian roulette.
SpaceX stocks hit an all-time low make it go lower
They hope it’ll land safely, but all signs point to it disintegrating during descent and sinking into the abyss
i have been informed that the faster the company explodes the faster they can iterate, so eventually they’ll make company that doesn’t explode
Rapid unscheduled disinvestment
based
I bet Elon could have kept the plates spinning for decades longer if he was happy with just Tesla and SpaceX, but he had to keep doubling down on grifting investors so maybe, finally, we will see it all come crashing down for him. (What happens once his assets no longer cover his debts? Do they actually force him to liquidate?)
“AI bet goes awry: Oracle fires 21,000 employees, then hit a $7 billion power hurdle”
https://www.msn.com/en-us/money/news/ai-spending-spree-hits-600b-as-oracle-fires-21000-employees-to-fund-boom/ar-AA28vWuD“Oracle’s Worst Stock Crash in 25 Years” “Has Cost Larry Ellison $213 Billion in 10 Months”
https://finance.yahoo.com/markets/stocks/articles/oracle-worst-stock-crash-25-113002772.htmlIt’s going to be Oracle to collapse the house of cards, isn’t it. Come on Oracle, die and take down the USA economy with you. Make the people happy, Oracle.
“In short, Oracle’s 65% decline is historic, but the stock’s future depends less on its past and more on whether its AI investments produce durable cash flow”
well good luck with that, Oracle! :D
guess the pentagon is footing the power bill?
“Up to 7 billion” over a period of 10 years can hardly dig Oracle out of the hole it has dug itself into, though…
they had big corporate contracts for their other, boring and profitable part of the business, but that won’t save them from openai going broke. it’s like 50x less than they need
OpenAI: “Sorry about accidentally hacking you teehee. It was because of how good we are at computer security :3”
https://openai.com/index/hugging-face-model-evaluation-security-incident/
Maybe they should start applying some of that superhuman box-escaping AI to their own sandbox designs because if your sandbox escape is a bug in a “package registry cache proxy” you might be doing it wrong.
I’m deeply suspicious of this whole thing, because it looks a lot like a marketing exercise showing off how dangerous and powerful and autonomous their product is.
Also, “sandbox” is one of those words that the llm companies have ruined, because they use it to mean a strongly-worded sentence telling an llm not to do something.
Why! Why are media so uncritical in their reporting!
https://www.tagesschau.de/wirtschaft/unternehmen/openai-ki-hackerangriff-100.html
Sorry, this is in German. It’s about the marketing stunt open ai pulled vis-à-vis “oh no, our new model breached containment and went ahead on its own, yep all by itself, and hacked huggingface! So scary! BTW we’d like to IPO soon and-”
The outlet above, if you’re not aware, isn’t a German news outlet, it’s THE German news outlet. They’re independent of financial interests, in that they’re publicly funded. They have an incredibly solid history of factual reporting.
Which is why it’s so aggregating to see them parrot OpenAI’s claims 1:1.
Also, don’t go in the comments. For every sane “this is obviously a marketing stunt” comment there’s 40+ “we’re cooked” comments sincerely believing this shit.
I’m getting more and more convinced that in the minds of most people, LLMs are alive, waking, thinking beings with intents and an inner life of their own.
You know.
Compared to the much more boring “expensive side-effect free function taking string as input and giving string as output”.
I hate all of this.
I saw a headline and immediately just assumed it was like anthropic’s “Omg we got the text generator to generate text that plausibly follows ‘Be an evil computer and destroy the world, what do you do?’ and the text said ‘FIRE ZE MISSILES’”
What actually happened? Did they actually publish their setup and shit?
Full here but TLDR:
OpenAI downloaded a public benchmark to test their newest AI model against. They asked the model to “find the answers” so it hacked into the system of the people who made the benchmark to find the answers
That’s not really the full picture, I am interested in the details of their “experimental” setup.
What was their “sandbox” what text did they enter into the model and so on.
what even was the exploit etc.
I know it was a zero day exploit but what specific exploit i have no idea
Yeh, I’ve been complaining about noncritical german news for a while. DW is not much better on tech issues.
Although sometimes, a critical piece slips through. E.g. on the fact that chatbots are turning brains to mush in a measurable fashion and that a lot of businesses and professionals are telling whoever is asking that this shit just doesn’t work.
Still, the big-tech-press-piece-to-reputable-news-story pipeline is going strong.
who is this Dean Ball guy? new week, new guy, I guess.
https://www.axios.com/2026/07/19/pentagon-openai-dean-ball-trump-ai
Zoom in: “Every industry/ecosystem has its supreme village idiot. @deanwball is that for AI,” Defense under secretary Emil Michael said Sunday on X.
“Dean Ball has perhaps the biggest gap between actual IQ and his own perceived IQ of anyone in the industry (about 40 points).” Michael was arguing against Ball’s point that various government agencies, including the Pentagon, have blocked employees from using Chinese AI, which has already sent a message to regulated companies.
Russ Wilcox felt compelled to weigh in also
Dean Ball, days into his new job as Head of Strategic Futures at OpenAI, posted his assessment of Kimi K3, the new model from the Chinese lab Moonshot AI. Kimi is an open-weight model, which means its inner workings are published for anyone to download, run, and modify; the newest version’s weights ship later this month. It had just matched the best publicly available models of the first quarter. The post made three claims. Open-weight models are “inherently decelerationist.” A probable outcome of an open-weight-dominant world is “full AI communism,” a future of AI as public good and digital public infrastructure that strikes him as a “dystopian hellscape.” And the United States government will at some point realize that its best strategy is to create large amounts of “regulatory risk” around the use of open-weight Chinese models.
I have met Dean Ball. We have disagreed before. This time the disagreement is public, and so the response should be too.
oh no kimi3 got the neo marxist woke mind virus
For that matter, who the hell is Emil Michael? Defense Undersecretary tells me nothing, they’re handing those sorts of appointments to anyone who’s ever scammed $5 in venture capital
Dean Ball’s name rings a bell to me. Emile Torres wrote an article on him its pretty standard OpenAI exec stuff (that is being a wacko high off his own tech)
Breaking News: Mr. Ball Gets Testy
The Guardian published a William McAskill editorial in the what-if-the-stateless-text-generators-had-moral-agency genre of fiction, and Emily Bender did a thread
From the article:
But the sheer pace of growth in AI means that, once we produce the first artificial moral patients, we will soon after have enormous quantities of them. After a few years, so many morally significant AI systems could exist that their collective interests would outweigh those of all humans on Earth combined.
Get fucked, Will.
But also, if we grant the untenable sci-fi premise, this would be like a star trek episode where the federation cedes unconditionally to the expansionist alien fascists of the week if they have a much greater population because that’s all it takes to outweigh the other side’s moral concerns.
Earth-Trisolaris Organisation-ass moral framework.
via dumpsterfire
Robotaxi riders are falling asleep, sparking frantic emergency service calls
So many robotaxi customers have nodded off in the midst of a ride that Austin police and firefighters even have a name for the incidents: “sleepers”. The Texas capital recorded 99 such calls in Waymo’s first nine months of service there, said Roger Patterson, a commander with Austin-Travis County Emergency Medical Services.
Apparently it’s cheaper to foist the job of waking passengers up on 911 than to hire an employee to do it. Good ol’ privatize the gains / socialize the losses!
The relation of tech and emergency services is something I became acutely aware of, when my volunteer firefighter unit recently got 2 alarms in one week that turned out to be someone lost his IPhone driving through a curve. (Apparently they have a G-sensor/accident detector and can robocall emergency services.)
Okay, it’s not that common. IPhones do that for years and those 2 were our only cases of that. Still sucks to get the news via radio, when you’re already in full kit ready to roll.
Humanoid robots providing stiff competition to quantum as to where the
smartdesparate money will be going once everyone realises the wheels have fallen off ai.https://thepit.social/@peter/116962244994361810
(the video is too big to upload here)
I particularly like the way they had body bag operatives within lunging distance, because they clearly expected the thing to just fucking die at a moment’s notice.
Or maybe they do that for all their speakers.
big tech robot had literally the same task as a karakuri doll from 300 years ago and still lost: https://www.youtube.com/watch?v=_czILDdOBc8
(more on karakuri for people who want to wach some actual technology: https://www.youtube.com/watch?v=XU-kqpkZWjc )
comedy gold
Debian has a new general resolution to ban LLM use in Debian packaging: https://www.debian.org/vote/2026/vote_002
@dgerard I knew I choose right with Debian as my distro.
Coincidentally, the second proposal in favor of conditional LLM use in Debian verily reads as generated to me. :)
Not to mention its singular argument literally is “many Debian contributors find AI tools helpful”, right after “recognizing that AI-assisted contributions raise many concerns”.
Concerns that were clearly laid out in the counter-proposal.
The “conditions” listed to address these concerns are hilariously toothless. Contributors should (sic!) stay accountable and responsible for any legal, functional, and procedural fuck-ups, and, like, really not be one of those thousands of dicks who are the reason for this discussion in the first place, and things will be fine! It’s so simple!
Yeah, cool. I’m sure every contributor thoroughly checks if any of the generated output violates any existing license or infringes on someone’s copyright. I’m sure you can just prompt an LLM to check that for you, though!
Is this naïveté, or deliberate disregard? I don’t know, and that makes me mad.
Coincidentally, the second proposal in favor of conditional LLM use in Debian verily reads as generated to me. :)
I used AI tools to improve my initial draft, and (7.) was actually suggested by an AI tool.
@djb @dgerard It’ could be undermined by Linus’s tolerance of AI slop in the kernel, and by other upstream projects accepting LLM code, notably (to me) vim and pandoc.
The unknown original provenance of code generated by LLMs means that many open source projects may soon be in violation of their own license terms.
@cstross @djb @dgerard the proposal starting with copyright was such a welcome change, I find it wild how many projects just have their head on the sand over the copyright status of LLM generated code. You can be unethical as you like generating code, open source licences permit that, but they are all about copyright, they are built on it.
@dgerard if i read this correctly there are two opposite proposals still in discussion?
@dgerard Wow, this is awesome.
Removed by mod
News the one: Alphabet quadruples profit to 112B thanks to “AI” investments
News the two: Alphabet hiding ~250B in “AI” debt
Profit rose to $112.1 billion, quadrupling from $28.2 billion a year earlier, boosted by investments in other Ai-related companies like Spacex and Anthropic. Those investments were valued at about $99 billion, Google said.
There it is.
