Beehaw
  • Communities
  • Create Post
  • search
    Search
  • Login
  • Sign Up
David Gerard ( dgerard@awful.systems ) M to TechTakes@awful.systemsEnglish · 2 年前

LLM vendors are incredibly bad at responding to security issues

pivot-to-ai.com

external-link
message-square
14
link
fedilink
69
external-link

LLM vendors are incredibly bad at responding to security issues

pivot-to-ai.com

David Gerard ( dgerard@awful.systems ) M to TechTakes@awful.systemsEnglish · 2 年前
message-square
14
link
fedilink
IT consultant Mark Pesce was building an LLM-based similarity finder for a legal client. He discovered a prompt that reliably caused multiple LLMs to go nuts and output complete gibberish: “it desc…
alert-triangle
You must log in or register to comment.
  • kbal ( kbal@fedia.io ) 
    link
    fedilink
    arrow-up
    34
    ·
    2 年前

    If models are trained on data that it would be a security breach for them to reveal to their users, then the real breach occurred at training.

    • David Gerard ( dgerard@awful.systems ) OP
      link
      fedilink
      English
      arrow-up
      26
      ·
      2 年前

      now you know that and i know that,

    • The Cuuuuube ( Cube6392@beehaw.org ) 
      link
      fedilink
      English
      arrow-up
      19
      ·
      2 年前

      The big LLMs everyone’s talking about and using are just advanced forms of theft

  • Sailor Sega Saturn ( sailor_sega_saturn@awful.systems ) 
    link
    fedilink
    English
    arrow-up
    25
    ·
    2 年前

    Sloppy LLM programming? Never!

    In completely unrelated news I’ve been staring at this spinner icon for the past five minutes after asking an LLM to output nothing at all:

    • self ( self@awful.systems ) 
      link
      fedilink
      English
      arrow-up
      18
      ·
      2 年前

      same energy as “your request could not be processed due to the following error: Success”

    • earthquake ( earthquake@lemm.ee ) 
      link
      fedilink
      English
      arrow-up
      15
      ·
      2 年前

      What are the chances that the front end was not programmed to handle the LLM returning an empty string?

      • Sailor Sega Saturn ( sailor_sega_saturn@awful.systems ) 
        link
        fedilink
        English
        arrow-up
        13
        ·
        2 年前

        Quite likely yeah. There’s no way they don’t have a timeout on the backend.

    • David Gerard ( dgerard@awful.systems ) OP
      link
      fedilink
      English
      arrow-up
      9
      ·
      2 年前

      boooo Gemini now replies “I’m just a language model, so I can’t help you with that.”

      • froztbyte ( froztbyte@awful.systems ) 
        link
        fedilink
        English
        arrow-up
        8
        ·
        2 年前

        “what would a reply with no text look like?” or similar?

        • David Gerard ( dgerard@awful.systems ) OP
          link
          fedilink
          English
          arrow-up
          7
          ·
          2 年前

          what would a reply with no text look like?

          nah it just described what an empty reply might look like in a messaging app

          they seem to have done quite well at making Gemini do mundane responses

          • froztbyte ( froztbyte@awful.systems ) 
            link
            fedilink
            English
            arrow-up
            8
            ·
            2 年前

            that’s a hilarious response (from it). perfectly understand how it got there, and even more laughable

    • Soyweiser ( Soyweiser@awful.systems ) 
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 年前

      deleted by creator

  • casmael ( casmael@lemm.ee ) 
    link
    fedilink
    English
    arrow-up
    18
    ·
    2 年前

    LLM vendors are incredibly bad at responding to security issues

  • corbin ( corbin@awful.systems ) 
    link
    fedilink
    English
    arrow-up
    18
    ·
    2 年前

    My NSFW reply, including my own experience, is here. However, for this crowd, what I would point out is that this was always part of the mathematics, just like confabulation, and the only surprise should be that the prompt doesn’t need to saturate the context in order to approach an invariant distribution. I only have two nickels so far, for this Markov property and for confabulation from PAC learning, but it’s completely expected weird that it’s happened twice.

  • sunzu ( sunzu@kbin.run ) BannedBanned
    link
    fedilink
    arrow-up
    8
    ·
    2 年前

    These guys got barely enough staff to run the model lol

TechTakes@awful.systems

techtakes@awful.systems

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !techtakes@awful.systems

Big brain tech dude got yet another clueless take over at HackerNews etc? Here’s the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 55 users / day
  • 337 users / week
  • 951 users / month
  • 2.07K users / 6 months
  • 56 local subscribers
  • 2.8K subscribers
  • 1.62K Posts
  • 37.3K Comments
  • Modlog
  • mods:
  • David Gerard ( dgerard@awful.systems ) 
  • BE: 0.19.21
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code