📝 Throwaway Note: What Happens to AI Training Now?
Recent AI advances have been largely powered by training powered by massive compute. The algorithmic equivalent to infinite interns working on a diet of espresso, Red Bull, cocaine, and Adderall.
But what happens now?
Compute time and equipment aren’t free, but until recently money essentially was (at least for large companies).
Revenue rolled in with (relatively) minimal effort spent on the core (/ legacy) generators. Interest rates meant credit lines were basically the same as debit accounts. And the production lines and supply chains were well oiled machines.
Then, COVID.
And now all of those circumstances have changed.
Are Tensorflow and Google Cloud important enough to Google’s bottom line that it’ll keep making custom chips for AI and burning through compute time to train?
Will Meta have the luxury of focusing on anything other than generating revenue via its ad platform?
Microsoft can probably make a case that these efforts are core to its business thanks to Azure and the cloud. Plus its long standing with enterprise-grade clients that might be making all of these same decisions for themselves.
What happens to Amazon as it takes body blows from European regulators while consumers stand on the precipice of an inflation-induced recession?
What is Apple going to do with all that cash? (Besides spend a chunk on a mysterious car project.) It has a history of squandering a lead (see: Siri, podcasts, etc?)
OpenAI will keep going as long as it can keep money coming in. But what happens to those funds as the above dominos start to fall?
Crypto is taking a beating, which could mean more compute resources for AI. Or it could mean the infrastructure providing companies get battered to the point that another supply squeeze happens.
The days of a tech company training a model for the human equivalent of years, showing up to a meeting, saying “something something AI something,” and getting showered with cash are over.
So what happens now?
—-
I stole the throwaway note idea from Marcelo Rinesi