• I mean, it’s an issue in general when talking about those big subjects. It’s like global warming, we keep talking about it as a future risk, but really, the crisis is already there. Talking about it as a future problem is just a good way to keep ignoring it… :/

    • Both are concerning, but as a former academic to me neither of them are as insidious as the harm that LLMs are already doing to training data. A lot of corpora depend on collecting public online data to construct data sets for research, and the assumption is that it’s largely human-generated. This balance is about to shift, and it’s going to cause significant damage to future research. Even if everyone agreed to make a change right now, the well is already poisoned. We’re talking the equivalent of the burning of Alexandria for linguistics research.

      • undefined> Both are concerning, but as a former academic to me neither of them are as insidious as the harm that LLMs are already doing to training data. A lot of corpora depend on collecting public online data to construct data sets for research, and the assumption is that it’s largely human-generated. This balance is about to shift, and it’s going to cause significant damage to future research. Even if everyone agreed to make a change right now, the well is already poisoned. We’re talking the equivalent of the burning of Alexandria for linguistics research.

        It reminds me of the situation with steel where post atomic weapons its tainted. It can’t be used for scientific tools or equipment. You have to find and use it from pre-atomic bombs. https://en.wikipedia.org/wiki/Low-background_steel

        There is going to be “low background data” in the future.

  • When I saw how coherent and consistent conservative style posting was on the chatgpt2 model, it made me wonder how many online conversations I’ve had over the preceding years were actually with an algorithim spitting platitudes and cliches exactly how a real life conservative would.