Most of the big names in power electronics are chasing the same challenge: Developing voltage regulators that can handle the huge currents required by GPUs and other kilowatt-class AI chips. But with lots of power comes lots of passive components. These DC-DC converters depend on increasingly large clusters of inductors and capacitors that convert, filter, and decouple the power delivered to the processor, helping maintain stable voltage and keeping current within the processor’s ripple envelope.

However, these passive components can hamper power delivery as much as they help it. They often occupy the most power-sensitive real estate on the PCB, forcing the DC-DC converter further from the load.

The added distance not only leads to higher power losses but also slows the system’s response to the large transient loads common with AI workloads. Moreover, the capacitors mounted under the board can crowd out vertical power delivery (VPD) solutions that could help tackle both issues.

One solution proposed by Saras Micro Devices is to combine the capacitors and inductors into a single module and integrate it into the processor's package substrate. By bringing the regulators, passives, and compute die closer, Saras said it can reduce impedance in the power delivery network (PDN), improve efficiency, and enable tighter voltage regulation.

To learn more about its STILE technology and where it fits into the AI power picture, we reached out to Shaun Bowers, the startup's SVP of Product Management.

When engineers talk about power delivery being a “bottleneck” for AI accelerators and other chips in the same high-performance ballpark, what do they mean?

The power “bottlenecks” that engineers talk about exist at multiple levels. At the highest level, the most fundamental issue is the imbalance between the rapidly rising demand for power to support and fuel the buildout of AI data centers and the less rapid buildout of grid infrastructure supporting this demand. This imbalance makes it imperative that every watt of power that is available and delivered to the AI data center is used in the most efficient way possible. 

The next level at which a power delivery bottleneck exists is within the power delivery network (PDN) itself, all the way from the grid to the chip. However, this issue is particularly amplified as you get closer to the device.

With thermal design power (TDP) increasing to multi-kilowatt levels and data center power approaching gigawatt levels, the resistive losses in the PDN and heat generated by these losses are becoming too significant to ignore, particularly in an environment where the infrastructure needed to deliver the required power is already constrained.

To sustain growth and establish an economically viable business, it’s now critical to maximize the amount of compute per watt and minimize the cost per operation and token. Accomplishing this requires rethinking the PDN architecture at every step to maximize efficiency and minimize the associated heat load and power required for cooling.

Power delivery losses can add up from the rack to the chip. Where are those losses typically happening? 

The losses from the rack to the chip occur during the various power-conversion steps within the PDN and from the resistive losses (I²R) associated with the distribution of that power to the chip. Today, most data center racks use a 54-V/48-V backplane as the primary power distribution system for the server, including the accelerator and processors.

On each accelerator and processor board, a 48- to 12-V voltage regulator module (VRM) is typically found along with a multitude of 12-V to point-of-load (POL) VRMs mounted on the board in a lateral arrangement around the chip that steps down from the 12-V input to the level required to service each of the power rails on the device.

As the level of silicon integration within the device goes up, the power density also increases to multiple kilowatts. At the same time, the advanced process nodes being used to fabricate the chips require lower operating voltages (0.6 to 0.8 V).

The net effect is that the amount of current that needs to be delivered by the PDN, from the VRM to the chip (I = P/V), is rising dramatically, and resistive losses in the PDN are growing exponentially with them.

Today’s systems rely on passive components in different locations to keep power stable, from the decoupling capacitors beneath the chip to bulk capacitors near the VRMs. Why do you need this hierarchy, and what are the limitations of the current approach?

The capability to correct voltage immediately during load changes is key to improving the system's overall performance. Traditionally, parallel capacitors are used along the power delivery path as sources of charge to mitigate current surges quickly and stabilize voltage levels across the full frequency range of interest.

Each capacitor type is characterized by its impedance and the frequency range for which it is most effective. So, a capacitor network made up of multiple capacitor types, rather than just one type, is more effective in reducing impedance.

Today, the system must deliver an increasing amount of current. As the currents increase, more capacitance is needed, resulting in the need for more “critical” board-level real estate for power conversion and power delivery. Even as converter switching frequencies rise and converter sizes are reduced, passives remain a limiting factor.

How does Saras’ STILE technology fit into the power delivery network? Where is it located?

The STILE technology is an integrated passive "module" that is purposely designed for embedding into IC package substrates and system PCBs (Fig. 1). Unlike traditional ceramic passive components used in surface-mount applications, Saras' capacitors are manufactured to match the thickness of the substrate where they’re going to be embedded. They’re fabricated with 3D pass-through copper terminations, exhibit almost no capacitance derating under temperature and bias, and are rated for operation at 125°C.

The solutions we provide today are primarily designed to service the output capacitance needs for the VRM, which helps stabilize and improve the PDN response and performance of the extreme transients occurring on today’s power-hungry processors. We have another generation of products under development that will allow us to handle higher-frequency applications with both decoupling capacitance and inductance.

What are substrate-embedded passives, and why are they important for package-level power delivery?

Rack-to-chip power delivery networks have traditionally used a lateral architecture where the VRMs are located on the top side of the PCB, encircling the chip. But again, the resistive losses associated with this architecture are becoming too big to ignore. So, what everyone wants to do is relocate these VRMs to the bottom of the PCB, directly in the shadow of the chip, to reduce the resistance of the power delivery path. 

Today, the underside of the PCB is where many of the surface-mounted capacitors supporting the PDN are located. To enable vertical power delivery, or VPD as everyone calls it, the passives need to be moved out of there. So, it makes sense to place the capacitors in the board or package substrate between the VRM and the load. Saras is aiming to address this by supplying "embeddable" passive solutions that work with VPD.

How does moving passive components closer to the load change the design options available to AI and HPC system architects?

With a shorter distance between the power supply and the silicon core, system architects are able to eliminate parasitic resistance and inductance, unlocking a lot more options across silicon, packaging, and board-level engineering (Fig. 2).

By being located nearer to the load, the transient response, system efficiency, and computational throughput are also all improved. This significantly reduces the energy cost per operation, enabling more operations per watt, which is a key metric for data centers.

What should engineers or system architects ask when evaluating package-level power delivery approaches?

Evaluating package-level power delivery solutions requires a multidisciplinary approach that involves working through complex electrical, thermal, and mechanical questions. While electrical parameters such as power, current density, and impedance are critical, they must be balanced against thermal and mechanical considerations and manufacturing tradeoffs.

Notwithstanding the reduction of power losses, increased integration in power delivery systems, particularly in package and in proximity to high-current-consuming SoCs, exposes every component in the power delivery path to high temperatures. Stability across the temperature range is important for the chip due to limitations in the operating temperature for the device. 

In addition, device packages are made of multiple materials that can have very different thermal and mechanical properties. The coefficient of thermal expansion (CTE) is one of these, and it boils down to how severely a material expands or contracts due to temperature change. Mechanical reliability and warpage minimization are both important given the level of heat in AI servers. Since warpage is temperature-dependent, performance must be evaluated and guaranteed across the entire reflow profile.

Implementing cooling solutions within a highly confined space presents a significant multi-physics challenge. Therefore, mitigating thermal challenges is complex and requires a system-level approach.

Looking ahead, how do you expect power delivery architectures to evolve as AI and high-performance-computing (HPC) devices continue to move to higher power levels?

It's obvious at this point that traditional approaches to power delivery are no longer going to keep up with the performance and efficiency needs of AI systems. As devices move to higher power levels, we believe that the need for higher efficiency and more integrated PDNs close to the load will continue to evolve. 

The first step is the transition from a lateral to a vertical PDN architecture. The next step will be to increase the voltage in the rack, and at each subsequent power-conversion step between the rack and the chip, to reduce the current and associated resistive losses at every step of the way. We also expect the PDN to further disaggregate and begin to incorporate in/on package and/or on-die integrated voltage regulators (IVRs) to further improve the efficiency and performance of the PDN. 

IVRs are still in the early stages of development and will need new architecture and active semiconductor device developments. VPD designs are expected to remain a vital part of the ecosystem as a front-end for lower-input-voltage IVRs that will ultimately be adopted as part of a vertical stackup.