Beehaw
  • Communities
  • Create Post
  • search
    Search
  • Login
  • Sign Up
 ryujin470   ( @ryujin470@fedia.io )  to Technology · 2 个月前

Researchers show that training on “junk data” can lead to LLM “brain rot”

arstechnica.com

external-link
message-square
9
link
fedilink
64
external-link

Researchers show that training on “junk data” can lead to LLM “brain rot”

arstechnica.com

 ryujin470   ( @ryujin470@fedia.io )  to Technology · 2 个月前
message-square
9
link
fedilink
Models trained on short, popular, and/or “superficial” tweets perform worse on benchmarks.
alert-triangle
You must log in or # to comment.
  •  hperrin   ( @hperrin@lemmy.ca ) 
    link
    fedilink
    English
    arrow-up
    12
    ·
    2 个月前

    Just like humans.

  •  jarfil   ( @jarfil@beehaw.org ) 
    link
    fedilink
    arrow-up
    12
    ·
    2 个月前

    Using a complex GPT-4o prompt, they sought to pull out tweets that focused on “superficial topics”

    Wait a moment… They asked an LLM, to tell them what was “junk”, and another LLM, trained on what an LLM marked as junk, turned out to be a junk LLM?

    It talks about model collapse, but this smells like research collapse.

  •  HappyFrog   ( @HappyFrog@lemmy.blahaj.zone ) 
    link
    fedilink
    arrow-up
    6
    ·
    2 个月前

    I say continue! It will be fun to have a brain rotted ai that can only quote memes

    •  SharkAttak   ( @SharkAttak@kbin.melroy.org ) 
      link
      fedilink
      arrow-up
      4
      ·
      2 个月前

      This explains the AWS collapse even better.

  •  Damage   ( @Damage@feddit.it ) 
    link
    fedilink
    arrow-up
    2
    ·
    2 个月前

    AI brain rots humans, itself, the internet… Just great

  •  VitoRobles   ( @VitoRobles@lemmy.today ) 
    link
    fedilink
    English
    arrow-up
    2
    ·
    2 个月前

    Is junk data just YouTube content because of so, then it’s already poisoned my kids and their friends.

    Whatever 67 means.

  •  Tollana1234567   ( @Tollana1234567@lemmy.today ) 
    link
    fedilink
    arrow-up
    2
    ·
    2 个月前

    no, training LLM on other LLM, on unverified facts, or sources leads to brain rot.

    •  darkmogool   ( @darkmogool@feddit.org ) 
      link
      fedilink
      Deutsch
      arrow-up
      3
      ·
      2 个月前

      same, same, but different, but still the same

  •  OpenStars   ( @OpenStars@piefed.social ) 
    link
    fedilink
    English
    arrow-up
    1
    ·
    2 个月前

    img

Technology

technology

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !technology@beehaw.org

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

  • Free and Open Source Software
  • Programming
  • Operating Systems

This community’s icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 155 users / day
  • 873 users / week
  • 2.79K users / month
  • 6.35K users / 6 months
  • 5.07K local subscribers
  • 41.1K subscribers
  • 5.51K Posts
  • 98.3K Comments
  • Modlog
  • mods:
  •  Chris Remington   ( @remington@beehaw.org ) 
  •  alyaza [they/she]   ( @alyaza@beehaw.org ) 
  •  TheRtRevKaiser   ( @TheRtRevKaiser@beehaw.org ) 
  •  gyrfalcon   ( @gyrfalcon@beehaw.org ) 
  •  rs5th   ( @rs5th@beehaw.org ) 
  •  coldredlight   ( @coldredlight@beehaw.org ) 
  •  Leigh   ( @SemioticStandard@beehaw.org ) 
  •  TheRtRevKaiser   ( @TheRtRevKaiser@kbin.social ) 
  • BE: 0.19.13
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code