AI can cheaply fake almost every signal we traditionally used to judge quality. I now see this pattern everywhere, in open source, in education, in research, all breaking at the same time. I find signaling theory useful for diagnosing what is happening and why it is so disorienting, and signaling theory and adjacent literature can provide the vocabulary to discuss possible solutions.

  • Monument ( Monument@piefed.world ) 
    link
    fedilink
    English
    arrow-up
    18
    ·
    7 hours ago

    Full disclosure, I have only skimmed the source article because I just woke up and am not quite with it, but your comment brought to mind two real-world situations that I’m currently trying to navigate.

    In my regular working life, I recently hired an assistant. The person we hired seemed like a great fit. They had all the requisite knowledge and skills represented in their resume and work history, with examples of actual work product to show for their efforts. Sadly, I’ve come to realize that all of their work and the terminology they threw around in their interview was from AI. When asked to use tools and systems that they cannot just dump into an AI system - you know, things that require actual intelligence and effort to wield - they flounder, plod along, and honestly, have a lower quality of work than most amateurs I interact with.
    It’s not just that they don’t seem to know the material, it’s that they don’t seem to be capable of learning it, either.

    In my extracurricular working life (my freelance work, side hustle, or what-have-you), I work with an educational institution to create educational materials for students. Very template-driven, and with some strict editorial and stylistic confines. Key amongst them is accessibility.
    Before I get my hands on the files, they go through a robust editorial process where they are placed in an organizational template that mirrors the layout file. Lately I’ve been receiving files that look like the template, but are just all the ‘Normal’ style in Word, with overrides applied. Which is to say that someone has been using AI to style the document, and the AI has not been applying headings, or other elements that define document structure - a key component of accessibility. We’re having lengthy discussions about this - both practical, and philosophical. On the philosophical side, we’re struggling with the decoherence of information. And I think that also factors into cognitive surrender - a term I very much like.

    (Thank you coming to my TED talk.)