Like everyone else we have been closely watching the developments in AI and writing about it, a lot about it. Here we want to sum up our overview of the sector with what we believe are the three important strategic questions for AI semis. These are the key areas of competition right now and how they play out will determine growth for the industry and many individual companies’ fortunes. We are only going to look at it from semis’ point of view, leaving aside some important questions about AI more broadly – including such questions as: How will consumers use AI? What business models will succeed with AI?; and Will AI Superbots wipe us all out?

• First, is AI additive to the semis market? Will purchases of AI chips increase overall demand for semis, or will they. merely replace purchases of other chips?
• How will the market for AI Inference semis play out? How big will it be and who will win share?
• Can Nvidia be displaced from the market dominance it currently enjoys?

The first question is very much on the industry’s minds, even if it often goes unspoken. Many companies, especially Nvidia, are seeing big jumps in demand for anything AI-related. There is fear in some quarters that customer budgets are fixed and so purchase of more GPUs to run AI models will come at the cost of traditional processors. From what we can tell this is not happening, instead it looks likely that the hyperscalers are increasing budgets in order to accommodate their AI needs. Anecdotally, we know that demand for new data center locations has spiked as companies seek to make room for all the GPUs they are buying. We think that with time this will temper and budgets will normalize, but for the next few years the industry can enjoy a strong AI purchasing cycle. Even more tantalizing is the trend of large enterprises building internal AI systems, reversing (or at least mitigating) a decade-long trend of migration to the cloud. This would be good news for many, especially start-ups.

The second question starts with the assumption that the market for Inference semis will be much larger than the market for Training chips and potentially an order of magnitude larger than the market today. Consumers have quickly become fascinated with generative AI, and while this may be wavering a bit, it is clear that even this initial demand is taxing the inference resources of all providers. As (If?) AI applications start to find broader consumer traction, the need for running inference will likely skyrocket. No one has really determined how much semis demand this will create, or at least no one really wants to talk about it for risk their CFOs hear these figures and pass out. This is going to be a major expense and our estimate is that the economics will only work out if much of the work gets done at the “edge”, on individual PCs and smartphones. For the industry, the AI Inference market is a fantastic opportunity. It will add to overall demand, and it is big enough to allow room for multiple entrants. The bad news is that most of those gains will likely accrue to vendors selling into data centers. Inference demand at the edge, while important, is much more likely to end up as a feature in existing product lines rather than incremental demand. Put another way, inference is likely to be additive to the data center semis market, but neutral to the market for edge semis.

The final question is Nvidia invulnerable? Nvidia’s current market lead is based on two factors. First, they have been investing heavily in AI for many years, and are now in the right place at the right time. It is better to be lucky than good, even better to be both. Second, their Cuda framework has proven instrumental in enabling the current AI momentum, but it now seems to have become a strategic barrier ‘locking’ customers into buying Nvidia semis. The combination of limited supplies and high prices for Nvidia GPUs has created massive incentives for customers to search for alternative GPUs, but as long as Cuda remains an important tool for optimizing AI deployments their choices may be limited. Set against this is the growing success of PyTorch and Triton, projects which provide alternatives to Cuda. In many ways, this looks like an old-fashioned fight over operating system and platform economics, and the outcome remains highly uncertain. Our sense is that market pressures are going to break Cuda’s hold on the industry, there is just too much friction with and reluctance to being dependent on a single chip vendor. That being said, the success of PyTorch does not mean doom for Nvidia. They are very busy building other software assets to cement their position in the market. In addition, there is the possibility that PyTorch ends up co-existing with Cuda. We think the industry, especially the hyperscalers, are going to take extreme measures to prevent a return to some modern version of the WinTel monopoly of the 1990’s, and dedicate significant resources to building out Cuda alternatives. This would be very good news for the broader industry, and not necessarily terrible news for Nvidia.

How big will the market for AI semis be? How will it be distributed across the landscape? And which companies will be the big winners? None of these are clear yet, but answering these questions will be the framework we use as we keep watch on the space.