Morgan Stanley just revised what it costs to build a frontier AI cluster, and the new numbers are reshaping which companies can actually compete in the AI arms race. The financial gap between winners and everyone else is widening faster than Wall Street anticipated.
    • nullachtfuenfzehn ( nullachtfuenfzehn@feddit.org ) 
      3·
      3 months ago

      I am looking forward to the day someone writes a piece of software that makes these massive data centers irrelevant as efficiency gains can be archived without it or more and more companies, institutions run open source models. Who the heck does really need a frontier model.

      • ExtremeDullard ( ExtremeDullard@piefed.social ) English
        3·
        3 months ago

        The fundamental problem with AI is that it breaks the principle of seeking economy of scale that’s been fueling tech for decades.

        AI simply doesn’t scale: in the best case scenario, a 2x performance requires 2x the resources to run the models. In practice, it’s even worse than that. It means like unlike previous techs that created a bubble, the massive capital investments don’t translate into cheaper operating costs, which don’t translate into more customers or increased profits.

        AI as it’s done today is a fucking dead end until someone invests in the fundamental research needed to run AI efficiently. But nobody does that because all the VC money is funneled into scaling up shit that doesn’t scale well.

        • kibiz0r ( kibiz0r@midwest.social ) English
          2·
          3 months ago

          You’re right on the money.

          Wading Through AI (series where an AI skeptic interviews an AI veteran who is deeply critical of the hype train) had an episode specifically on scale: https://youtube.com/watch?v=bgWq678Oed4

          The main thing, the nugget that I want people to keep in mind is that the expected performance is roughly logarithmic [with respect to] the inputs. … So what that means is that … to get additive performance, you have to double the amount of compute and the amount of data.

      • kibiz0r ( kibiz0r@midwest.social ) English
        1·
        3 months ago

        Who the heck does really need a frontier model.

        Almost nobody. In my experience, there are basically two good ways to use AI:

        1. When you have a well-defined problem for which a potential answer can be validated programmatically. Basically, “I need to get here; I don’t care how, just let me skip the friction of getting there”.
        2. When you have a poorly-defined problem and need help looking at the problem from multiple new angles. Basically, adding friction so that you can understand something better. Crucially, this is about improving your understanding, not producing a final output.

        The workflows that AI companies are trying to sell will supposedly go from poorly-defined problem to provably valid solution. But that’s a really bad idea, for so many reasons. It takes a ton of power, it’s unreliable, and it can fail spectacularly.

        Hank Green had a quip a while ago, something like “I have a hope that we’ll look back at this time period and be like: remember when the big AI companies had everyone basically driving Formula 1 cars everywhere, when they really just needed a Honda Civic?”