We spent much of the past week trying to size the market for AI servers in the data center. We are not sure we are any closer to having an answer to that question, but we are definitely several steps closer to madness. This is a post about being an analyst and how we get our numbers, not for the squeamish, there is no shame in turning away now before seeing how the sausage is made.

For most people, the default source for this kind of information are the large tech industry analyst firms like IDC and Gartner. Those reports are expensive, and there is a low price alternative in the form of Taiwan’s Digitimes. We have reviewed data from all three and already the problems start to emerge. First, Digitimes does not have a great reputation among many. They have what could politely be called a different set of editorial standards, and many find their data unreliable. On the other hand, they are very close to the relevant companies who are actually building these servers – the Taiwan based ODMs. In practice, this means there is a big disconnect between the Digitimes figures and the US firms’ numbers. Sometimes the difference is as much as 40%, but it varies, which means they are not just counting things differently. And all of this data is constantly changing, with each month’s updates reflecting changes not only to the forecasts but to the historical numbers as well. IDC and Gartner’s numbers are much closer to each other, but nor are they in complete agreement.

Another concern is that the growth rates do not look right. We will not publish their numbers here, even though some of them can be found on the public web. Speaking in round numbers, one of the estimates we saw says the market for AI servers was roughly 1.6 million units in 2023 and will grow to 1.8 million units in 2024, roughly 10% growth. How do we square that figure with Nvidia’s results? Their latest quarter data center compute revenues grew 478% year-on-year. Most AI servers today are powered by Nvidia, so those numbers should be more closely correlated.

Here we have to dig into a the definition of an AI server, which of course is fraught. The analysts define an AI server as basically any type of data center server not powered by a CPU which means GPUs and AI accelerators (largely Google TPUs). AI servers differ from traditional servers in that they tend to be larger in volume (so fewer fit on a rack) but denser in processors – up to 8 GPUs per server. So this ratio of GPU per server is important, but no one can quite agree on just what the ratio is on average. We used data from a couple firms and weighted averaged them out to around something like 3.5 GPUs per AI server. On this metric, the growth in content is more like 200%, better than 10% growth, but still far short of what Nvidia is reporting. Factor in price increases and the fact that Nvidia’s revenue is boosted by the prices it charges for HBM memory (which is a big amount), and maybe we are in the ballpark, or at least playing the same sport.

That being said, our sense is that there is just a big disconnect between what the analysts are reporting and what we are seeing in the market. We also took a look at a half dozen sell-side research reports. These numbers match up a bit better to the market, and that really gets us to the heart of the problem.

We know many analysts who work at the big firms. They are smart, hard working people looking to do honest work, but our sense is that they face structural issues with how they conduct their work. Gathering data on this industry is difficult. For a large part it relies on self-reporting from the vendors. One of the reasons Digitimes can sell its research is that they have those insights into the ODMs based a few high speed rail stops away. However in the US, the silicon vendors are often an important source of data. Nvidia has never had close relationship with the analyst class for historical reasons beyond our scope today, which means that AMD and Intel are leading data sources. Those companies have built-in reasons for undercounting the market – the larger the market, the smaller the share of AMD and Intel looks internally. Again, we think the analysts putting this data together are doing their best, honest work. And we do no think anyone at the big companies is being deceitful. It’s just that everyone is making estimates and assumptions on the market size, and this can lead to mis-estimation. Extrapolating all this out just makes it worse. Who is building the servers? Who is buying them? How does that change over the next ten years? We do not know the answer yet, but our tin foil hat should be ready soon, and that will help us get the answer.