• self ( self@awful.systems ) 
    link
    fedilink
    English
    arrow-up
    52
    ·
    11 months ago

    Copilot then listed a string of crimes Bernklau had supposedly committed — saying that he was an abusive undertaker exploiting widows, a child abuser, an escaped criminal mental patient. [SWR, in German]

    These were stories Bernklau had written about. Copilot produced text as if he was the subject. Then Copilot returned Bernklau’s phone number and address!

    and there’s fucking nothing in place to prevent this utterly obvious failure case, other than if you complain Microsoft will just lazily regex for your name in the result and refuse to return anything if it appears

      • self ( self@awful.systems ) 
        link
        fedilink
        English
        arrow-up
        29
        ·
        11 months ago

        god, so this is actually the best the AI researchers can do with the tools they’ve shit out into the world without giving any thought to failure cases or legal liability (beyond their manager on slackTeams claiming it’s been taken care of)

        so fuck it, let’s make the defamation machine a non-optional component of windows. we’ll just make it a P0 when someone who could actually get us in legal trouble complains! everyone else is a P2 that never gets assigned.

              • V0ldek ( V0ldek@awful.systems ) 
                link
                fedilink
                English
                arrow-up
                12
                ·
                11 months ago

                How do you measure good/bad at predicting words? What’s the metric? Cause it doesn’t seem to be “the words make factual sense” if you’re defending this.

                • self ( self@awful.systems ) 
                  link
                  fedilink
                  English
                  arrow-up
                  12
                  ·
                  11 months ago

                  like fuck, all you or I want out of these wandering AI jackasses is something vaguely resembling a technical problem statement or the faintest outline of an algorithm. normal engineering shit.

                  but nah, every time they just bullshit and say shit that doesn’t mean a damn thing as if we can’t tell, and when they get called out, every time it’s the “well you ¡haters! just don’t understand LLMs” line, as if we weren’t expecting a technical answer that just never came (cause all of them are only just cosplaying as technically skilled people and it fucking shows)

                • MagicShel ( MagicShel@programming.dev ) 
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  edit-2
                  11 months ago

                  No. Predicting words is barely related to facts. I’ll defend AI as an occasionally useful tool, but nothing it ever says should be taken as fact without confirmation. Sometimes that confirmation can be experimental — does this recipe taste good? Sometimes you need expert supervision to say this part was translated wrong or this code won’t work because of xyz. Sometimes you have to go out and look it up.

                  I like AI but there is a real problem treating it like the output means anything. It might give you a direction to look closer at, but it can never be the endpoint. We’d be better off not trying to censor it, but understanding it will bullshit you without blinking.

                  I summarize all of that by saying AI is a useful tool, but a terrible product.

    • Soyweiser ( Soyweiser@awful.systems ) 
      link
      fedilink
      English
      arrow-up
      16
      ·
      11 months ago

      lazily regex

      I’m having a sneaking suspicion that this is what they do for all the viral ‘here the LLM famously says something wrong’ problems, as I don’t think they can actually reliably train the model it made an error.

      • MagicShel ( MagicShel@programming.dev ) 
        link
        fedilink
        English
        arrow-up
        11
        ·
        11 months ago

        That’s the most straightforward fix. You can’t actually fix the output of an LLM, so you have to run something on the output. You can have it scanned by another AI but that costs money and is also fallible. Regex/delete is the most reliable way to censor.

        • Soyweiser ( Soyweiser@awful.systems ) 
          link
          fedilink
          English
          arrow-up
          11
          ·
          11 months ago

          Yes, and then the problem is that this doesn’t really scale well. Esp as it is always hard to regexp all the variants correctly without false positives and negatives. Time to regexp html ;).

          • OhNoMoreLemmy ( OhNoMoreLemmy@lemmy.ml ) 
            link
            fedilink
            English
            arrow-up
            8
            ·
            11 months ago

            Yeah, and you can really see this in image generation. There’s often blocks on using the names of celebrities in the prompts, but if you misspell the names enough it can bypass the censor, and the image generator still understands it.

  • superkret ( superkret@feddit.org ) 
    link
    fedilink
    English
    arrow-up
    14
    ·
    11 months ago

    Microsoft published, using their software and servers, a libelous claim, to potentially millions of people.
    The details of how the software was programmed should be legally irrelevant.