Much of the conversation about compute and AI today ends up being a conversation about the public cloud providers. These hyperscalers build multiple data centers a year and consume somewhere north of 50% of the $50 billion market for data center semiconductors. There are less than a dozen of them, and as big customers, they make for easy analysis. But there is still the other half of the market, and half of $50 billion is still a large market. Right?

This market is particularly important for start-ups looking to design new chips. Selling into the hyperscalers is a time-consuming process, and to put it mildly, the hyperscalers have a poor track record when it comes to supporting start-ups. Start-up executives have come to realize pursuing the hyperscalers is a perilous journey and so many have started to pin their hopes on penetrating the Enterprise data center market.

This should be a solid strategy. The market has a big customer base comprised of customers who are large enough to afford real compute budgets, but not so large as to be overwhelming. There are a lot of targets, and it only takes a few wins to build a revenue stream large enough to support a start-up.

Unfortunately, this does not seem to be the case. For starters, the trend is still moving strongly towards the “Cloud”. The big enterprises are still moving more of their workloads to someone else’s data centers. Another problem is that while these companies are large, they may not be large enough to see the value in moving to some new type of chip. Even the hyperscalers, with their immense scale and control of much of their own software stack, take on new vendors reluctantly. The costs of porting and optimizing software to new chips is significant, especially in terms of corporate IT time, In practice, this means that start-ups going this route need to dedicate significant sales engineering (Field Application Engineers or FAEs) to supporting the few customers they can bring in. This makes scaling a small chip business significantly harder, not insurmountable, but definitely a throttle on growth.

New chips even from established vendors, need to demonstrate performance improvements to merit purchase. Factor in the costs of learning all the intricacies of a new company’s idiosyncrasies means the performance needs to be that much better, and demonstrating that level of advantage is incredibly challenging.

Fortunately, there are some factors that may alter this balance. First and foremost is the growing awareness of ‘data sovereignty’, or who owns your data. This has always been important to enterprises when contemplating the move to the Cloud, but often a good contract and some straightforward security practices have been enough to assuage these concerns. Which brings us to AI. It is one thing to run corporate email servers in the Cloud, where basic encryption and terms of service eliminate most vulnerabilities, but it is a whole other thing to put a company’s most sensitive data and plans into AI systems which are very much a black box even to its creators. There is considerable hope that AI will reverse the trend for moving ‘everything’ to the Cloud. Another more prosaic factor is economics of the Cloud. For companies of sufficient size, running most IT on-premise is probably the best value for money. (Martin Cassado at A16Z has written a lot about this.) Coupling these together and there is a growing argument for large enterprises to move more of their compute back home.

Admittedly, the biggest beneficiary of that trend would be incumbents AMD, Nvidia and Intel, as it does not solve the very real problem of porting software and learning a new stack. But it does at least leave the door open for new entrants to this market.

Photo by Alessandro Erbetta on Unsplash