I can’t wait for companies to finally price out most of developers out of AI use, especially the FOSS ones.
I just hope most of them won’t get too addicted to the tech crack they are getting free/cheap samples of currently, and will be able able to find back their motivation and skill to work without a feel-good dopamine machines.
Also, lol at all the coments being like “if you’re 100% against the tech crack, you’re delusional. The cat is already out of the bag, it makes you way better at coding, if you use it responsibly!”
The problem isn’t that it’s not somewhat good, the issue is that soon you won’t be able to afford it, while also being addicted and dependant on it. But I’m sure y’all are able to use crack responsibly and will be fiiine.
you get free electricity or your PC parts are made of magic? running inference workloads frequently is going to have a significant impact. This is on top of using your PC for other things like gaming and work.
What hardware that needs? My issue with running local models was that it’s too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.
Mac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn’t free, but at least it’s just a single upfront payment.
If the project is understaffed and mistakes were made, wouldn’t it be more constructive to help maintain it or encourage broader participation, rather than dogpiling on a volunteer maintainer?
Even if too expensive for FOSS devs the mega corps relying on their software will still be able to afford them to run their own security testing, feeding the bug reports back to the project. And with time the hardware and models are only getting more efficient (for a comparable performance level).
I can’t wait for companies to finally price out most of developers out of AI use, especially the FOSS ones.
I just hope most of them won’t get too addicted to the tech crack they are getting free/cheap samples of currently, and will be able able to find back their motivation and skill to work without a feel-good dopamine machines.
Also, lol at all the coments being like “if you’re 100% against the tech crack, you’re delusional. The cat is already out of the bag, it makes you way better at coding, if you use it responsibly!”
The problem isn’t that it’s not somewhat good, the issue is that soon you won’t be able to afford it, while also being addicted and dependant on it. But I’m sure y’all are able to use crack responsibly and will be fiiine.
I run Qwen 3.6 27B at home. For “free”. It is extremely useful.
My point being that I’m not going to be priced out of using it
most people are going to destroy their home servers running these workloads
Destroy as in the fan bearings are going to wear out quicker?
you get free electricity or your PC parts are made of magic? running inference workloads frequently is going to have a significant impact. This is on top of using your PC for other things like gaming and work.
What hardware that needs? My issue with running local models was that it’s too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.
Geforce 3090 with 24TB should be able to run a “Q5 version” of it. Maybe get a second older computer, or maybe you can run two cards in one PC.
64GB unified memory. I run it (and a lot more) on a dgx spark, but a Mac mini would suffice also.
You could prob run 4-bit version on a RTX card with 32g. Maybe even 24g. Like a 5090 or 4090 or such.
So much info out there.
Mac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn’t free, but at least it’s just a single upfront payment.
Don’t worry, they want to replace your hardware with a “cloud based computing solution” as well.
When did that absurdity come back? I thought we killed the cloud computer nonsense a decade ago.
Well you see… subscriptions.
If the project is understaffed and mistakes were made, wouldn’t it be more constructive to help maintain it or encourage broader participation, rather than dogpiling on a volunteer maintainer?
Even if too expensive for FOSS devs the mega corps relying on their software will still be able to afford them to run their own security testing, feeding the bug reports back to the project. And with time the hardware and models are only getting more efficient (for a comparable performance level).
deleted by creator