Over the holidays we got a chance to test out Google’s new “Night Sight” , this is the capability available on the company’s Pixel phones which takes incredible low-light photos. They unveiled the service a few months ago, you can read about it here. We have to admit that we were very impressed. We mention it here because it says something important about semiconductors and the evergreen theme of software eating the world. Put simply, software advances are greatly outpacing chip advances in many important ways.

For us, the incredible thing about Night Sight is that Google is able to do this entirely with software. As everyone knows, Google is one of the leaders in Machine Learning and has been pushing the technical envelope on all forms of ML-powered imaging tech.

This really struck a chord with us because over the years we have encountered probably a dozen hardware companies trying to do new things with digital imaging and cameras. We would guess that over the past decade venture investors have burned something like $500 million on chip imaging start-ups. And established chip companies have probably  spent double that. As analysts, we saw many of these companies. All of that investment has now been largely rendered obsolete by advances in software.

Let’s look at an example to explore this. Five years ago, we were invited to sit on a panel judging a start-up demo day. The event took place in the conference room of a very fancy, very well-known Sand Hill Road law firm. The audience consisted of what we would describe as amateur angel investors – retired executives, lawyers and doctors who wanted to participate in the early stages of the latest tech Bubble. By far, the best received company presenting that day was a company that claimed to have solved the camera ‘bump’ problem. Apple had just launched their latest iPhone, the first with that odd-looking bump on the back of the phone housing the camera module. This company claimed to have solved the problem and the audience ate it up. This company had not sold a single product (and as far as we know never did). It had only very early engineering samples and no clear manufacturing plan. Despite those ‘minor’ problems, the conversation in the room quickly turned to more ‘important’ questions like “How much will Apple pay to acquire this company?”. Having just come from a company that struggled to support as Apple as a customer, we knew this M&A talk was ludicrous. But the audience didn’t want to hear about that, they just wanted to know who to make out the checks to.

The camera bump exists because of the laws of physics. Camera quality is in part determined by the distance between the camera lens and the image sensor. To get sufficient quality requires a minimum distance, and so the lens has to extend out the back of a phone, creating the bump. There is no easy  way to fix this in hardware. Like we said, there is that matter of the laws of physics.

Phones makers could remove the bump, but that would lead to lower quality cameras, and cameras are clearly  a killer app for smartphones, the center of intense competition and brand definition. That being said, if you could combine multiple (two or three) low quality cameras and aggregate their images with smart software, you could remove the bump and still get great quality photos. We know of another chip start-up that tried to this, but ultimately  failed because they did not have sufficient expertise in software to pull it off.

Which of course brings us back to Google. They seem to have accomplished this great trick. They have mastered image enhancement with ML software. To be clear, Night Sight is available via a software update to many existing phones, implying that no hardware requirements other than a good smartphone camera. And while they have not yet explicitly announced a multiple camera upgrade to photo quality, they appear to be trialing something that looks very similar. At this point, we expect the next generation of Pixel phones to take advantage of this. The current version of the Pixel has a modest bump, but it is a little thicker than the latest iPhone, so the likely change would be a noticeably thinner phone with no bump. We have no idea if Apple has anything similar in the works. More precisely, we do not know if they were working on something similar, but we would wager that now they definitely are.

We have written a lot lately (and here) about how the large chip companies are falling behind the chip capabilities of their largest customers. And while the demand the for chips will never go away, this trend looks likely to further de-value the R&D efforts of those chip companies. The management of the big chip companies face a serious problem. Not only are companies like Apple and Amazon building their own key chips, but the chip incumbents lack the software expertise needed to compete in the future.