So Far, AI Is a Money Pit That Isn't Paying Off
So Far, AI Is a Money Pit That Isn't Paying Off
While the stock market may love the term "AI," companies haven't figured out how to optimally monetize the services that go along with it.
Silicon Valley has bet big on generative AI but it’s not totally clear whether that bet will pay off. A new report from the Wall Street Journal claims that, despite the endless hype around large language models and the automated platforms they power, tech companies are struggling to turn a profit when it comes to AI.
Microsoft, which has bet big on the generative AI boom with billions invested in its partner OpenAI, has been losing money on one of its major AI platforms. Github Copilot, which launched in 2021, was designed to automate some parts of a coder’s workflow and, while immensely popular with its user base, has been a huge “money loser,” the Journal reports. The problem is that users pay $10 a month subscription fee for Copilot but, according to a source interviewed by the Journal, Microsoft lost an average of $20 per user during the first few months of this year. Some users cost the company an average loss of over $80 per month, the source told the paper.
OpenAI’s ChatGPT, for instance, has seen an ever declining user base while its operating costs remain incredibly high. A report from the Washington Post in June claimed that chatbots like ChatGPT lose money pretty much every time a customer uses them.
AI platforms are notoriously expensive to operate. Platforms like ChatGPT and DALL-E burn through an enormous amount of computing power and companies are struggling to figure out how to reduce that footprint. At the same time, the infrastructure to run AI systems—like powerful, high-priced AI computer chips—can be quite expensive. The cloud capacity necessary to train algorithms and run AI systems, meanwhile, is also expanding at a frightening rate. All of this energy consumption also means that AI is about as environmentally unfriendly as you can get.
Good. Maybe the hype will finally die down soon and "AI" won't be shoved into every nook, cranny, and Notepad app anymore.
I'll say AI is a bit more promising, but all of this just really reminds me of the blockchain craze in 2017. Every single business wanted to add blockchain because the suits upstairs just saw it as free money. Technical people down below were like "yeah cool, but there's no place for it". At least I could solve some problems, but business people again just think that it's going to make them limitless money
Nah, blockchain has extremely limited applications.
Generative AI legitimately does have quite a number of areas that it can be made use of. That doesn't mean that it can't be oversold for a given application or technical challenges be disregarded, but it's not super-niche.
If you wanted to compare it to something that had a lot of buzz at one point, I'd use XML instead. XML does get used in a lot of areas, and it's definitely not niche, but I remember when it was being heavily used in marketing as a sort of magic bullet for application data interchange some years back, and it's not that.
Technical people down below were like “yeah cool, but there’s no place for it
I think you might underestimate entertainment and creation. Right now I can imagine some character or scenario in my head and generate a little avatar with stable situation then render it onto a live chat that (mostly) works.
I've paid like 2k for a computer that enables this. It's make money from me at least.
AI has been paying of for decades, it is used in all industries for appropriate tasks.
Now it is even better we are doing stuff no one thought it could be possible and advancing our work.
Perfect use case is for things that are simple to do but take too much time to be economical for humans (ex. counting products, plants, trees, cars, disease detection...) and using additional data to make better decisions.
Generative AI (for writing text, coding,... ) is of course no where close to being useful, but it can interesting to try. It is just a toy, expensive one, but still a toy.
Generative AI (for writing text, coding,… ) is of course no where close to being useful, but it can interesting to try. It is just a toy, expensive one, but still a toy.
I disagree. It's absolutely useful... in certain industries, for appropriate tasks.
That doesn't stop it from also being used as a toy.
for appropriate tasks.
100% agree with this. Good, sharp knives are perfect for creating wonderful dishes in the kitchen. But people can also use them for harm.
I've seen some incredibly useful ways in which gen-AI can be used. A few months or so back, there was someone here on the Fediverse that was able to spin up a working prototype of a self-hosted Spotify replacement after only 8 or so hours of gen-AI code development. Stuff that would've taken 5x as long to research and code with just a human.
Gen-AI is ultimately a useful technology... when used the right way.
AI is absolutely taking off. LLMs are taking over various components of frontline support (service desks, tier 1 support). They're integrated into various systems using langchains to pull your data, knowledge articles, etc, and then respond to you based on that data.
AI is primarily a replacement for workers, like how McDonalds self service ordering kiosks are a replacement for cashiers. Cheaper and more scalable, cutting out more and more entry level (and outsourced) work. But unlike the kiosks, you won't even see that the "Amazon tech support" you were kicked over to is an LLM instead of a person. You won't hear that the frontline support tech you called for a product is actually an AI and text to speech model.
There were jokes about the whole Wendy's drive thru workers being replaced by AI, but I've seen this stuff used live. I've seen how flawlessly they've tuned the AI to respond to someone who makes a mistake while speaking and corrects themself ("I'm going to the Sacramento office -- sorry, no, the Folsom office") or bundles various requests together ("oh while you're getting me a visitor badge can you also book a visitor cube for me?"). I've even seen crazy stuff like "I'm supposed to meet with Mary while I'm there, can you give me her phone number?" and the LLM routes through the phone directory, pulls up the most likely Marys given the caller's department and the location the user is visiting via prior context, and asks for more information - "I see two Marys here, Mary X who works in Department A and Mary Y who works in Department B, are you talking about either of them?"
It's already here and it's as invisible as possible, and that's the end goal.
This is just what is visible to users/customers which is just top of the iceberg.
Real use of AI is in every industry and best use case is for jobs that were imposible before.
This article isn't saying that AI is a fad or otherwise not taking off, it absolutely is, but it's also absolutely taking too much money to run
And if these AI companies aren't capable of turning a profit on this technology and consumers aren't able to run these technologies themselves, then these technologies may very well just fall out of the public stage and back into computer science research papers, despite how versatile the tech may be
What good is a ginie if you can't get the lamp?
it’s also absolutely taking too much money to run
Well, maybe they should raise their prices then?
If they raise the prices too far though, I'll just switch to running Facebook's open source llama model on my workstation. I've tested and it works with acceptable quality and performance, only thing that's missing is tight integration with other tools I use. That could (and I expect will soon) be fixed.
This is how investments in new technologies work. That's such a non-story.
The GitHub copilot example seems to indicate it’s a pricing problem. In fact this situation might indicate that users are finding it so useful that they are using it more than MS expected when they set up their monthly subscriptions. Over time, models are going to be optimized and costs will reduce.
Expecting AI to take over all human intensive tasks is not realistic but eventually it’s going to become part of a lot of repetitive tasks. Though I hope that we see more open source base models instead of the current situation with 3-4 major companies providing the base models behind most of the AI applications.
Even with the costs implied here, Copilot would be useful as hell.
Think about it, an average (western) developer costs easily 100k/year, sometimes even 2 or 3 times that. Spending something like 1000€ per year makes sense, even if productivity is increased by just one percent.
GitHub Copilot is extremely useful. It also runs pretty much on every key stroke and programmers make a lot of keystrokes throughout the day...
It's a useful enough tool that people would be willing to pay more, but at the same time it's not using an advanced AI model. It uses an older (and now deprecated) model that I'm pretty sure a high end computer (even some laptop) could provide similar output without any cloud service, using open source / freely available models.
My feeling is Copilot needs to either lower their price or improve the quality of the product if it's going to survive. And I suspect they're going to do the latter one.
The other factor not discussed here is the hardware we use today for this task isn't really designed for it. The GPUs most datacentres run AI models on are designed for graphics, not AI, and the algorithms mostly just need huge amounts of fast memory. I'm sure soon there will be dedicated hardware specifically designed for large language models with less compute cores and more memory. They'll likely also run at lower clock speeds and use less power/generate less heat/etc.
Just because companies are losing money right now doesn't mean they will be in five years time.
I’m sure co-pilot will be revamped with the newer GPT-models, they’re just not prioritizing it right now.
Okay... let's call wine "wine" and bread "bread": the acronym "AI" is mostly an advertisement stunt.
This is not artificial intelligence; and even if it was, "AI" is used for a bag of a thousand cats - game mob pathfinding, chess engines, swarm heuristic methods, so goes on.
What the article is talking about is far more specific, it's what the industry calls "machine learning"¹.
So the text is saying that machine learning is costly. No surprise - it's a relatively new tech, and even the state of art is still damn coarse². Over time those technologies will get further refined, under different models; cost of operation is bound to reduce over time. Microsoft and the likes aren't playing the short game, they're looking for long-term return of investment.
- I'd go a step further and claim here that "model-based generation" is more accurate. But that's me.
- LLMs are a good example of that; GPT-4 has ~2*10¹² parameters. If you equate each to a neuron (kind of a sloppy comparison, but whatever), it's more than an order of magnitude larger than the ~1*10¹¹ neurons in a human brain. It's basically brute force.
The comparison of GPT parameters to neurons really is kinda sloppy, since they're not at all comparable. To start with, "parameters" encompasses both weights (ie. the "importance" of a connection between any two neurons) and biases (sort of the starting value of an individual neuron, which then biases the activation function) so it doesn't tell you anything about the number of neurons, and secondly biological neurons have way more dynamic behavior than what current "static" NNs like GPT use so it wouldn't really be surprising if you needed much more of them to mimic the behavior of meatbag neurons. Also, LLM architecture is incredibly weird so the whole concept of neurons isn't as relevant as it is in more traditional networks (although they do have neurons in their layers)
Another sloppiness that I didn't mention is that a lot of human neurons are there for things that have nothing to do with either reasoning or language; making your heart beat, transmitting pain, so goes on. However I think that the comparison is still useful in this context - it shows how big those LLMs are, even in comparison with a system created out of messy natural selection. The process behind the LLMs seems inefficient.
But ML is being used in the industry in tons of places, and it's definitely cost effective. There's simple models that take the input of machinery sensors and detect when something is faulty or needs repairing, not just malfunctioning parts but worn out parts too. It's used heavily in image processing, tiktok is used by a lot of people and the silly AR thingies use image recognition and tracking in real time through your phone. I'm not saying that this features created revenue directly, but they do get viral and attract users, so yeah. Image processing is also used in almost any supermarket to control the amount of people in the store, at least since covid I see a dynamically updated counter in every supermarket I visit.
It is also used in time estimations, how much traffic influences a trip is not manually set at all, it gets updated dynamically with real time data, through trained models.
It is also used in language models, for real usages like translation, recommendation engines (search engines, store product recommendation...).
The article is talking about generative models, more specifically, text prediction engines (ChatGPT, Copilot). ChatGPT is a chatbot, and I don't see a good way to monetise it while keeping it free to use, and Copilot is a silly idea to me as a programmer since it feels very dangerous and not very practical. And again, not something I would pay for, so meh.
But (+> contradiction) ML is being used in the industry in tons of places [...] store product recommendation…).
By context it's rather clear which type of machine learning I'm talking about, given the OP. I'm not talking about the simple models that you're talking about and that, as you said, already found economically viable applications.
Past that "it's generative models, not machine learning" is on the same level as "it's a cat, not a mammal". One is a subset of the other, and by calling it "machine learning" my point was to highlight that it is not a "toothed and furry chicken" as the term AI implies.
The article is talking about generative models
I'm aware, as footnote #1 shows.
As someone who has always been skeptical of "AI," I definitely hope corporations dial back their enthusiasm on it; but I think its value has never been commercial, but industrial.
"AI" was not designed so consumers could see what it would look like to have Abraham Lincoln fighting a T-Rex without having to pay artists for their time. "AI" was designed so that could happen on a much larger enterprise scale (though it would probably be stock images of technology or happy people using technology instead).
With this in mind, I think "AI" being a money pit won't dissuade corporations since they want the technology to be effective for themselves, they just want consumers to offset costs.
Exactly. It will lead to improved automation for industrial processes but it won't ever be a consumer tech other than improving your siri results.
It won't replace jobs, anymore than industrial robots in factories replace them.
“AI” was not designed so consumers could see what it would look like to have Abraham Lincoln fighting a T-Rex without having to pay artists for their time.
Sure... AI can do that... but it can also be used for "here's photo of my head with trees in the background, remove the trees".
I could also do that as a human, but it'd take me hours to do a good job blending my hair into a transparent png without a green tinge. AI can do it in seconds.
Just because a tool can be used to do useless things, doesn't mean the tool is useless.
Billionaires spend billions making some jobs just a bit more efficient, against the wishes of the people who do those jobs, and then cry when it doesn't actually pay that much.
'big data' 'crypto' etc.
AI is just the next 'big thing' that amounts to nothing. 90% of what anyone says in the press/media is total nonsense. And most AI researchers are downplaying the hype because they know it's all bullshit and AI will ultimately not be a major change anymore than navigation systems in cars was. It is merely convenience and those that 'rely' on it will end up in trouble.
It's a complementary technology, not a revolution.
I feel like companies were all hoping to get in early, to get a solid chunk of the cake. Well, and then a lot more companies got in than anyone could have guessed, so the slices of the cake are a lot smaller.
We'll have to see what happens, though. It's possible that the startups have to give up and only a few big fish remain. But if those have to increase prices to become profitable, this market will still be a lot smaller than people were hoping for.
Good, we as a society aren't ready for these kind of tools. AI would further increase the divide between people. One of the reasons is that it costs too much to run it.
Everyone who can't afford the hardware would be dependent on AI owned by corporations. And most people can't even afford those fees. Since we build our society around (materialistic) "productivity", I am sure AI would escalate how we treat people and whole countries who fall off the capitalism train. I hope the hype dies.
Ok, but AI isn't going away. So if these companies stop serving open access, the ONLY people that will use them will be the people who can afford the server/processing time.
This article isn't about usefulness of the models to normal people. It's about profitability of the models to the corporations that serve them.
Rather than debate every new technology for energy worthiness on a case-by-case basis up front, it would be more productive to direct our efforts towards better policy that internalizes the cost of pollution.
"These new-fangled 'lasers' don't even do anything useful for the energy they consume!"
I'm not too well-versed on the subject but, isn't user interactions with LLM's also train them further? They make it sound like the product has already been matured and they're letting people use it for free.
Good
🤖 I'm a bot that provides automatic summaries for articles:
Click here to see the summary
A new report from the Wall Street Journal claims that, despite the endless hype around large language models and the automated platforms they power, tech companies are struggling to turn a profit when it comes to AI.
Github Copilot, which launched in 2021, was designed to automate some parts of a coder’s workflow and, while immensely popular with its user base, has been a huge “money loser,” the Journal reports.
OpenAI’s ChatGPT, for instance, has seen an ever declining user base while its operating costs remain incredibly high.
A report from the Washington Post in June claimed that chatbots like ChatGPT lose money pretty much every time a customer uses them.
Platforms like ChatGPT and DALL-E burn through an enormous amount of computing power and companies are struggling to figure out how to reduce that footprint.
To get around the fact that they’re hemorrhaging money, many tech platforms are experimenting with different strategies to cut down on costs and computing power while still delivering the kinds of services they’ve promised to customers.
Saved 60% of original text.
Oh, no! Billionaires with short term, selfish thinking might lose money! What a tragedy.
With MS hinting at SAAS for Win12 with Copilot, this will be a good test.
Nobody can profit off "monthly usage credits" or without a subscription.
At least not until the tech becomes more affordable.
Could OpenAI refactor their code and algorithms to be more efficient? The more Instructions Per Cycle (IPC) that can be performed in the same amount of time can reduce costs to some extent when employed at scale.
That and computers are always getting more energy efficient, so if they can have these keep up or outpace user growth then it might become a little more sustainable.
No one cares about openai any more
My guess is specialized hardware. Similar to how mining crypto moved from CPU -> GPU -> ASIC, we'll see LLM hardware that optimizes the currently-cpu-intensive pieces.
Although you are right that more specialized hardware will come, NVIDIA has those already, and probably other manufacturers too. I'm not saying there isn't a lot room for improvement, but most of these AI models are simply matrix multiplications, and we are good at those.