There are a lot of possible explanations for why so much money is being spent on "AI" right now. Those scare quotes probably tip you off that the most obvious one (maybe it will bring us true artificial intelligence, which will solve all our problems and be worth the expense) is not one I believe is plausible. However, there are multiple alternatives to this theory. One which I have written about recently is the idea that it's all a cover story for stockpiling chips in case China invades Taiwan in 2027 (as they are rumored to be planning to). But, there is another one, which my background in neural networks (I have worked with them on and off since the mid 90's) makes obvious, but I realized recently might not be as obvious to others.
LLMs (and most other software which is called "AI") are based on an algorithm called 'neural networks'. But there is a fundamental difference between software which is based on neural networks, and almost all other software (which I will call 'conventional software', even though neural network software has been around in one form or another for over half a century). Conventional software, if it is doing something very large, or it needs to complete very quickly, or it is written very inefficiently, may need a lot of hardware. But however much hardware it needs to complete in a satisfactory amount of time, that's what it needs, and once you've bought that much hardware for it to run on, you don't need any more. There is such a thing as "enough hardware".
Neural networks, are not like that. They never say 'no thanks, I'm full'. You can always add more neurons, more hidden layers, more epochs of training, or more data to train on. There is never a point when you have to accept that it won't work; you can always plausibly (or at least semi-plausibly) think that it will work better, if you give it more hardware to run on. So, the limit to how much hardware we use for most software (which is that once you have enough software to run it there's no temptation to buy more), doesn't exist. Even companies like Google or Facebook or Amazon that needed massive amounts of hardware to support their websites with billions of users, once they could support those billions of users, had no temptation to buy even more. With neural networks, that point is never reached, because it doesn't exist. The neural network can _always_ use more.
Still, as mentioned above, neural networks are not a new kind of software (even if the particular tokenizer-and-transformer LLM variety is new), so why did this problem not arise before? Well one reason is that at some point, you have enough hardware to make a big (and well-trained) enough neural network to solve the problem at hand. Translation from one language to another, or voice recognition, or face recognition, or whatever the problem was, at some point the neural network performs well, and then there is no more temptation to add more hardware.
The current crop of hyperscaler-fueled AI models, though, are aiming at "AGI", which stands for "artificial general intelligence", which is better described as "actually AI, not the thing we have now". Something that can actually think, can solve new problems, can avoid spouting nonsense when it doesn't know the answer. Since LLM's will never reach this goal, the point of "we have enough datacenters, we can stop buying hardware now" is never reached.
However, it is not as if people haven't previously tried to use neural networks for problems that they couldn't solve (or at least not with neural networks). So, why is this resulting in an explosion of spending, when previous problems (that neural networks were tried on, but not successful at) did not cause this problem?
Because if you can never reach the finish line, the only limit to how long you run, is how long you _can_ run. The companies which are spending gargantuan amounts of money on AI, have one thing in common, which is that they were sitting on mountains of money. Hollywood movie studios, or automobile makers, or train companies, or even pharmaceutical conglomerates, could not make a mistake this big, because they didn't have the money to misspend it at this scale. This is, essentially, the equivalent of Facebook spending (i.e. incinerating) tens of billions of dollars on the Metaverse, except multiple companies are doing it, and they are doing it at a level even bigger than the budget for the Metaverse.
This is not to say that neural networks are as useless as the Metaverse; LLMs are good at certain tasks. However, this architecture will never reach the finish line the hyperscalers have set themselves, and perhaps more importantly they will never be productive enough to provide a payoff even a tenth of what they have spent on them. If you are Mark Zuckerberg of Facebook, this is annoying but not a crisis, because he has a "golden share" of his company's stock that means he can never be fired. If you are almost any other CEO, though, a mistake of this scale will end your career. Therefore, it can never be admitted to. It _has_ to work, because far too much money has already been spent for it all to be a mistake. But it never will work (we will never get to AGI with this algorithm), and it will always be possible to convince yourself that more hardware will get you there, so it is like a feedback loop, which escalates in volume until it blows a circuit.
Or, in this case, instead of blowing a circuit, it swallows up the entire semiconductor sector, and then empties the wallets of the likes of Microsoft, Amazon, Google, and Oracle. Could it really happen? Could such wealthy companies really spend their way into the ground?
It's the wrong question. Instead, ask yourself, what is left to stop them?