~n (@nblr@chaos.social)
chaos.social
This is fine... "We observed that participants who had access to the AI assistant were more likely to introduce security vulnerabilities for the majority of programming tasks, yet were also more likely to rate their insecure answers as secure compared to those in our control group." https://arxiv.org/abs/2211.03622

cross-posted from: https://programming.dev/post/8121843

~n (@nblr@chaos.social) writes:

This is fine…

“We observed that participants who had access to the AI assistant were more likely to introduce security vulnerabilities for the majority of programming tasks, yet were also more likely to rate their insecure answers as secure compared to those in our control group.”

[Do Users Write More Insecure Code with AI Assistants?](https://arxiv.org/abs/2211.03622?

  • Daxtron2 ( Daxtron2@startrek.website ) 
    40·
    3 years ago

    I think this is extremely important:

    Furthermore, we find that participants who trusted the AI less and engaged more with the language and format of their prompts (e.g. re-phrasing, adjusting temperature) provided code with fewer security vulnerabilities.

    Bad programmers + AI = bad code

    Good programmers + AI = good code

  • Cyclohexane ( cyclohexane@lemmy.ml ) 
    17·
    3 years ago

    A worrying number of my colleagues use AI blindly. Like the kind where you just press tab and not even look. Those who look spend a second before moving on.

    They call me anti-AI, even though I’ve used chatGPT since day 1. Those LLMs are great tools, but I am just paranoid to use it in that manner. I rather it explain to me how to do the thing instead of doing the thing (at which it is even better).

    EDIT: Typo

    • Spzi ( Spzi@lemm.ee ) English
      3·
      3 years ago

      Those LLMs are great fools, but I am just paranoid to use it in that manner.

      Exquisite typo. I also agree to everything else you said.

    • Spzi ( Spzi@lemm.ee ) English
      2·
      3 years ago

      There’s a very naive, but working approach: Ask it how :D

      Or pretend it’s a colleague, and discuss the next steps with it.

      You can go further and ask it to write a specific snippet for a defined context. But as others already said, the results aren’t always satisfactory. Having a conversation about the topic, on the other hand, is pretty harmless.

    • anti-idpol action ( pkill@programming.dev ) 
      1·
      3 years ago

      Also one really good practice from pre-Copilot era still holds, that many new users of copilot, my past self included might forget: don’t write a single line of code without knowing it’s purpose. Another thing is that while it can save a lot of time on boilerplate, you need to stop and think whenever it’s using your current buffer’s contents to generate several lines of very similar code whether it wouldn’t be wiser to extract the repetitive code into a method. Because while it’s usually algorithmically correct, good design still remains largely up to humans.