Rimu ( rimu@piefed.social ) to Programming@programming.devEnglish · 4 months agoRsync author responds to online outrage about his usage of LLMsmedium.comexternal-linkmessage-square132linkfedilinkarrow-up1180file-textcross-posted to: programmingprogramming@lemmy.mlpulse_of_truth@infosec.pubmain@0xdd.org.ru
arrow-up1180external-linkRsync author responds to online outrage about his usage of LLMsmedium.comRimu ( rimu@piefed.social ) to Programming@programming.devEnglish · 4 months agomessage-square132linkfedilinkfile-textcross-posted to: programmingprogramming@lemmy.mlpulse_of_truth@infosec.pubmain@0xdd.org.ru
Seems like he’s been pushed into using LLMs as a way to cope with the deluge of LLM-generated security reports.
minus-squarelocuester ( locuester@lemmy.zip ) linkfedilinkEnglisharrow-up1·4 months ago64GB unified memory. I run it (and a lot more) on a dgx spark, but a Mac mini would suffice also. You could prob run 4-bit version on a RTX card with 32g. Maybe even 24g. Like a 5090 or 4090 or such. So much info out there.
minus-squarewewbull ( wewbull@feddit.uk ) linkfedilinkEnglisharrow-up1·4 months agoMac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed. Running local isn’t free, but at least it’s just a single upfront payment.
64GB unified memory. I run it (and a lot more) on a dgx spark, but a Mac mini would suffice also.
You could prob run 4-bit version on a RTX card with 32g. Maybe even 24g. Like a 5090 or 4090 or such.
So much info out there.
Mac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn’t free, but at least it’s just a single upfront payment.