Who can make money from inference chips?

Latest update time：2024-01-09

Reads：

????If you want to see each other often, please mark it as a star???? and add it to your collection~

Source : Content compiled by Semiconductor Industry Observation (ID: ic bank ) , compiled from techspot, thank you.

An important topic in semiconductors today is the recognition that the real market opportunity for AI chips will be the AI inference market. We think it makes sense, but we're starting to wonder if anyone is going to make money from it.

The AI inference market is important for two reasons. First, Nvidia seems to have a lock on AI training. Granted, both AMD and Intel have products in this space, but we'd classify those as "ideal" right now. Currently, this is Nvidia's market. Second, the AI inference market is likely to be much larger than the training market. Intel CEO Pat Gelsinger has a great analogy for this – weather models. Only a few entities need to create weather forecast models (NASA, NOAA, etc.), but everyone wants to see the weather.

The same is true for artificial intelligence, where the utility of a model will depend on the ability of the end user to use the model. So the importance of the inference market has been a consistent theme at all the analyst and investor events we've attended recently, and even Nvidia has recently changed their positioning to talk more about inference.

Of course, there are two parts to the inference market - cloud and edge. Cloud inference happens in the data center, edge inference happens on the device. We've been hearing people debate the definitions of the two lately, and the lines can get a little blurry. But we think that if the companies operating the model pay the capex for cloud inference, and if the end users pay the capex for edge inference (by buying a phone or a PC), then the segmentation is fairly simple.

Cloud inference is probably the most noteworthy competition. Nvidia makes a very strong case for why they will shift dominance from training to inference. In short, there is a lot of overlap, and Nvidia has Cuda and other software tools that make connecting between the two very easy. We suspect this will appeal to many customers, we're in a "buy Nvidia and you won't get fired" era, and the company has a lot to offer here.

On the other hand, their big competitors will fight very hard for this market share. Additionally, hyperscalers, which likely consume the majority of inference chips, have the ability to break their dependence on Nvidia, whether by designing their own chips or leveraging competitors. We expect this to be a focus of much attention in the coming years.

The edge inference market is a more open question. First, no one really knows how much AI models will rely on the edge. Companies operating these models (especially hyperscale enterprises) really want edge inference to dominate. This will significantly reduce the amount of money they need to spend building all these cloud inference data centers. We suspect that, if not possible, the economics of AI may not become obsolete.

That said, this vision comes with an important caveat – are consumers really willing to pay for artificial intelligence? Today, if we asked the average consumer how much they would pay to run ChatGPT on their computer, we suspect the answer would be $0. Yes, they are willing to pay $20 per month to use ChatGPT, but are they willing to pay more to have it run locally. The benefits of this are not entirely clear, maybe they can get answers faster, but ChatGPT is already pretty fast when delivered from the cloud. If consumers are unwilling to pay more for a PC or phone with "artificial intelligence capabilities," chipmakers won't be able to charge a premium for chips with those features. We mentioned that Qualcomm faces this problem in the smartphone space, but the same applies to Intel and AMD in the PC space.

We've asked everyone about this but haven't received a clear answer yet. The reality is that we don’t know what consumers are willing to pay because we don’t really know what AI will do for consumers. When pressed, the semiconductor industry executives we spoke with all tended to default to some version of "We've seen some incredible demos coming soon" or "We think Microsoft is working on some incredible matter". These are all fair answers, we're not full-blown AI skeptics (yet) and we imagine there are some incredible projects underway.

This reliance on software raises the question of how valuable artificial intelligence will be to semiconductor manufacturers. If the value of these AI computers depends on software companies (especially Microsoft), then it's tempting to assume that Microsoft will capture the majority of the value of consumer AI services. Microsoft is an expert in this area. The only push for AI chips will likely be to trigger one-time device updates. That will pay dividends in a year or two, but it's much smaller than the huge opportunity some companies have to build artificial intelligence.

Original link

https://www.techspot.com/news/101432-opinion-anyone-going-make-money-ai-inference.html

Click here to follow and lock in more original content

END

*Disclaimer: This article is original by the author. The content of the article is the personal opinion of the author. The reprinting by Semiconductor Industry Watch is only to convey a different point of view. It does not mean that Semiconductor Industry Watch agrees or supports the view. If you have any objections, please contact Semiconductor Industry Watch.

Today is the 3639th issue shared by "Semiconductor Industry Observation" with you. Welcome to pay attention.