Ex-CISA chief says AI could mean the end of cybersecurity
www.theregister.com
: Jen Easterly says most breaches stem from bad software, and smarter tech could finally clean it up
    • NuXCOM_90Percent ( NuXCOM_90Percent@lemmy.zip ) 
      27·
      1 year ago

      I mean, at a high level it is very much the concept of ICE from Gibson et al back in the day.

      Intrusion Countermeasures Electronics. The idea that you have code that is constantly changing and updating based upon external stimuli. A particularly talented hacker, or AI, can potentially bypass it but it is a very system/mental intensive process and the stronger the ICE, the stronger the tools need to be.

      In the context of AI on both sides? Higher quality models backed by big ass expensive rigs on one side should work for anything short of a state level actor… if your models are good (big ol’ “if” that).

      Which then gets into the idea of Black ICE that is actively antagonistic towards those who are detected as attempting to bypass it. In the books it would fry brains. In the modern day it isn’t overly dissimilar from how so many VPN controlled IPs are just outright blocked from services and there is always the risk of getting banned because your wifi coffee maker is part of a botnet.

      But it is also not hard to imagine a world where a counter-DDOS or hack is run. Or a message is sent to the guy in the basement of the datacenter to go unplug that rack and provide the contact information of whoever was using it.

    • notarobot ( notarobot@lemmy.zip ) 
      3·
      1 year ago

      A few years back someone made virus that connected to an llm server and kept finding ways to infect computers in the simulated network. I think it was kind of successful. Not viable for a virus though, but an interesting idea non the less

  • Lucy :3 ( 30p87@feddit.org ) 
    55·
    1 year ago

    Genius strategy:

    • Replace Juniors
    • Old nerds knowing stuff die out
    • Now nobody knows anything about programming and security
    • Everything’s now a battle between LLMs
  • Mikina ( Mikina@programming.dev ) 
    23·
    1 year ago

    I have worked as a pentester and eventually a Red Team lead before leaving foe gamedev, and oh god this is so horrifiying to read.

    The state of the industry was alredy extremely depressing, which is why I left. Even without all of this AI craze, the fact that I was able to get from a junior to Red Team Lead, in a corporation with hundreds of employees, in a span of 4 years is already fucked up, solely because Red Teaming was starting to be a buzz word, and I had passion for the field and for Shadowrun while also being good at presentations that customers liked.

    When I got into the team, the “inhouse custom malware” was a web server with a script that pools it for commands to run with cmd.exe. It had a pretty involved custom obfuscation, but it took me lile two engagements and the guy responsible for it to leave before I even (during my own research) found out that WinAPI is a thing, and that you actually should run stuff from memory and why. And I was just a junior at the time, and this “revelation” got me eventually a unofficial RT Lead position, with 2 MDs per month for learning and internal development, rest had to be on engagements.

    And even then, we were able to do kind of OK in engagements, because the customers didn’t know and also didn’t care. I was always able to come up with “lessons learned”, and we always found out some glaring sec policy issues, even with limited tools, but the thing is - they still did not care. We reported something, and two years ago they still had the same bruteforcable kerberos tickets. It already felt like the industry is just a scam done for appearances, and if it’s now just AIs talking to the AIs then, well, I don’t think much would change.

    But it sucks. I love offensive security, it was really interresting few years of my carreer, but ot was so sad to do, if you wanted to do it well :(

  • onlinepersona ( onlinepersona@programming.dev ) 
    18·
    1 year ago

    I tried using AI in my rust project and gave up on letting it write code. It does quite alright in python, but rust is still too niche for it. Imagine trying to write zig or Haskell, it would make a terrible mess of it.

    Security is an afterthought in 99.99% of code. AI barely has anything to learn from.

    • krooklochurm ( krooklochurm@lemmy.ca ) 
      23·
      1 year ago

      If you’re using Hannah Montana Linux you can just open a terminal and type “write me ____ in the language ____” and the Hannai Montanai will produce perfectly working code every time.

    • wiegell ( wiegell@feddit.dk ) 
      2·
      1 year ago

      Mitchell Hashimoto writes a lot of Zig with AI (and this interview is almost a year old), see: https://www.youtube.com/watch?v=YQnz7L6x068&t=490s How long since you have tried tools? I think there has been some pretty astounding progress during the last couple of months. Until recently i did not use it daily, but now I just cant ignore the efficiency boost it gives me. There are definitely security concerns, and at this point you should not trust code that you do not read/understand, but tbh. i’m starting to believe that AI might (at least in the short term) free up resources to patch stuff and implement security features, that otherwise was not prioritised before due to focus on feature development. What it does to the IT sector in the long run - who knows…

      • onlinepersona ( onlinepersona@programming.dev ) English
        1·
        1 year ago

        That video showed him saying that it’s good for autocomplete. But speaking from experience testing it on Rust, Python, JS, HTML and CSS, it performed the worst on Rust. It wrote tests well, but sucked at features or refactoring. Whether the problem is between the chair and the screen, I don’t know.

        Whether AI will be able to write secure code, I dunno, I haven’t tried. It could be put into the rules to consider security and add tests relating to security or add an adversarial agent that tries to find flaws in the code which can be exploited. That could probably do more than a developer who has no time assigned to care about testing, much less security.

        What it does to the IT sector in the long run - who knows…

        Agreed. Things are moving so quickly, it’s impossible to predict. There are lots of people on LinkedIn screaming about obsoletion of humans or other bold claims, but to me they are like drunk fortune tellers: tell enough fortunes and one is bound to be right.

        • wiegell ( wiegell@feddit.dk ) 
          2·
          1 year ago

          My naive hope is that local models or maybe workplace distributed clusters catch up and the cloud based bubble bursts. I am under the impression, that atm. a big difference as to whether a tool works well or not is more related to how well all the software around the actual llm is constructed. E.g. for discovery - being able to quickly ingest an internet url and access a web index is a big force of the cloud based providers atm. And for coding it has a lot to do with quickly searching and finding the relevant parts of the codebase and evaluate whether the llm has all the required information to correctly perform a task.