Is the AI Market Already Consolidating?
Just about every processor-chip hardware vendor has at least one device that incorporates some sort of artificial-intelligence (AI) hardware acceleration in its portfolio. This might be a system-on-chip (SoC) with integrated AI acceleration or a dedicated AI accelerator that’s typically optimized for a particular application like processing streaming video.
A lot of AI startups are also vying for the engineer’s mindshare (Fig. 1). Many provide hardware solutions that significantly reduce system power requirements, allowing for more sophisticated edge-computing solutions that handle voice and video in real-time without passing things to the cloud.
Jon Peddie notes that “Over $13.5 billion has been invested in the startups since 2006. The level of excitement and investment in AI processor startups peaked from 2017 to 2021, with 36 new companies entering the segment.” You can find more details in the Jon Peddie Research "AI Processors Market Development Report." It also notes that fewer new companies are entering this market and there’s a consolidation happening, with vendors snarfing up AI companies to improve their own AI offerings.
Jon also notes that, “Currently, the U.S. maintains a strong position in both AI software and hardware. But that lead isn’t guaranteed. The ongoing trade tensions are putting real pressure on U.S. companies, especially now that firms like DeepSeek and several Chinese AI chipmakers are showcasing their capabilities.”
He adds, “U.S. suppliers are well aware of the situation, but the current administration appears to be focused elsewhere. That leaves these companies largely on their own, needing to move faster, think leaner, and deliver more efficient and cost-effective technologies to stay ahead.”
The U.S. administration’s latest gift to China was allowing NVIDIA to sell its H20 GPGPU to China. The H20 is theoretically less powerful than the company’s high end, “Hopper” H200 (Fig. 2) that targets large language models (LLMs). Multiple H200 GPGPUs can be connected via NVLink, which expands to larger configurations using NVLink switches.
What many overlook in NVIDIA’s success is that its software is more important than the hardware. True, the software doesn’t work without the hardware, but it enables engineers and programmers to target everything for the Jetson Nano, from edge computing to server-based solutions for the cloud and data center.
On the server side, the parts scale by adding more to a networked mix, so the only difference between something like a mix of H20s and H200s is the overall performance and capacity. Just add more hardware to increase these, which is why even allowing the H20 to be sold isn’t really a limitation for AI development.
Focusing on the Edge
Much of the discussion about AI focuses on the cloud and data center, where the H20 and H200 reside. However, a significant trend is developing in putting more AI compute at the edge, either offloading the use of cloud-based AI or completely decoupling with all of the compute being done locally. Quite a few of the AI startups are focusing on this area.
Small language models (SMLs) as well as systems that can handle LLMs on the edge are available now. The usual engineering design tradeoffs crop up in terms of power, price and performance, but new AI hardware has significantly changed the design equation.
As with chip vendors, board and module vendors have incorporated AI into their product mix. This facilitates adding AI at the edge, as integration of AI chips requires a more demanding design and implementation process.
With all of the AI hype, engineers and companies need to keep in mind that not all problems need AI or will be improved by adding AI in a product. Likewise, there’s a difference between using AI for your work, in your workflow, in the design, test, and production process, and in the final product and service. AI in the form of LLMs or SLMs might work in one area but not another.
Below is a quick poll on AI on the edge that I hope you'll take. It focuses on the target platform. The focus should simplify your answer. It will provide feedback on how everyone may be using AI in products that will deploy in the field and what we will be covering in the future.
