What Is Inside An AI Chip? Dies, HBM Stacks And Packaging, Explained In 3D
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: What Is Inside An AI Chip? Dies, HBM Stacks And Packaging, Explained In 3D on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

A November 22, 2026, explainer from ThorstenMeyerAI.com uses 3D models to show how AI accelerators combine compute dies, stacked high-bandwidth memory and advanced packaging. The article argues that memory bandwidth and the capacity to manufacture complex packages can constrain AI systems alongside compute performance.

ThorstenMeyerAI.com published a 3D explainer on November 22 showing how an AI accelerator is assembled from compute dies, stacked high-bandwidth memory (HBM), a silicon interposer, a package substrate and a cooling plate. The report traces those components into rack-scale systems, where Nvidia’s NVL72 designs contain 72 GPUs, and says memory bandwidth and advanced packaging are among the constraints shaping AI hardware supply and performance.

The article’s 3D models show the package from the bottom up: a package substrate connects the accelerator to its module; a silicon interposer carries dense wiring between the compute dies and memory; and the compute dies sit alongside HBM stacks. A cold plate is fitted above the package to carry liquid cooling. The models use the layer order and counts described in product and technical documents, but their dimensions are schematic: the article says Nvidia and AMD do not publish the relevant component measurements and that it does not estimate them.

HBM is made by stacking DRAM chips and linking them vertically with through-silicon vias, or TSVs. The memory towers sit close to the processor dies, allowing many connections within the package. The article says Nvidia lists 19.2 terabytes per second of memory bandwidth for Rubin, compared with 900 gigabytes per second for the V100. Those figures are product specifications for different generations, not a measure of application performance by themselves.

The package designs vary by manufacturer and generation. According to Nvidia’s architecture materials cited in the report, Rubin has two reticle-limited compute dies and 336 billion transistors. AMD describes its MI350-series design as eight vertically stacked accelerator complex dies and two I/O dies; the MI355X has 185 billion transistors. AMD says the forthcoming MI455X and MI450X use stacked 3D hybrid-bonded compute dies. The article also cites TSMC’s CoWoS packaging technology, which provides the interposer connections needed to bring compute and HBM together.

At a glance
reportWhen: Published November 22, 2026
The developmentThorstenMeyerAI.com published a 3D hardware explainer showing the layers inside AI accelerators and how 72-GPU systems are assembled.

Why Memory and Packaging Matter

The arrangement matters because an accelerator’s compute units need a steady supply of data. The report’s central point is that performance is not determined by compute dies alone: memory bandwidth affects how quickly data reaches those dies, while advanced packaging makes the high-density connections possible. Increasing the number or height of memory stacks and expanding interposer capacity are engineering and manufacturing challenges, not just choices in chip design.

Packaging capacity also has consequences beyond an individual processor. The report cites SK hynix’s expectation that HBM demand will exceed supply through 2030. That is the memory maker’s forecast, not an independently established timeline for every supplier or product. If demand does outpace available supply, it could limit how many complete accelerators can be assembled, even where compute dies are available. For buyers, that can affect system availability and cost; the precise effect depends on production, demand and supply-chain conditions.

At rack scale, cooling and interconnects become part of the system design. Nvidia says a GB200 NVL72 rack contains 72 GPUs and 36 Grace CPUs, distributed across compute trays and connected through NVLink switch trays. The report says Nvidia specifies 120 kilowatts of cooling capacity for the GB200 rack. Such figures help show why the relevant unit for deploying AI can be a fully integrated rack rather than a standalone chip.

Products Worth Considering

Jackery Solar Generator 1000 v2 with 200W Solar Panel, 1024Wh LiFePO4 Battery

Jackery Solar Generator 1000 v2 with 200W Solar Panel, 1024Wh LiFePO4 Battery

As an affiliate, we earn on qualifying purchases.

From Package Layers to Rack Systems

The article places current accelerator packaging in a progression toward larger and more complex assemblies. A reticle is the maximum area a chipmaking scanner can expose in one pass. TSMC describes its CoWoS-S interposer as supporting areas up to about 2,700 square millimeters, or 3.3 reticles. For larger packages, CoWoS-L connects smaller silicon bridges within a carrier. The report cites TSMC’s chief executive as saying on October 15, 2026, that the company was in real-world production of five-reticle-size packaging and working on nine- and 14-reticle versions.

HBM stacks are also changing. The report cites SK hynix’s description of stacking 12 layers of 3GB DRAM chips at the same total thickness as an earlier eight-layer product, with each DRAM chip made 40% thinner. Samsung has described a seven-micrometer gap between chips in its 12-high HBM3E. JEDEC’s HBM4 standard, published in April 2025, allows stacks of four, eight, 12 or 16 layers and up to 64GB per stack.

The jump from package to rack adds power, cooling and network equipment. Nvidia’s GB200 and GB300 NVL72 configurations are described as having 18 compute trays, nine NVLink switch trays, management switches and power shelves. The report cites an Open Compute Project listing that gives the GB200 rack dimensions as 600 millimeters wide, 1,068 millimeters deep and 2,236 millimeters high. These configurations are not interchangeable with every AI system, and the explainer’s schematic model is not a dimensionally exact rendering.

“Where AI gets stuck today, it is often not the compute die. It is memory bandwidth, packaging, and how many of these parts the world can actually build.”

— Thorsten Meyer, ThorstenMeyerAI.com

Limits of the 3D Model

The models are schematic rather than dimensionally exact, and the article says the companies do not publish the measurements needed to establish exact die, package or HBM-stack sizes. Counts and technical descriptions are drawn from cited product materials, but configurations can differ by product and system. The article also does not independently verify future supply levels or quantify how any HBM shortage would affect prices, delivery schedules or customers.

Some cited specifications refer to products at different stages or generations. The report describes Rubin as listed by Nvidia as available now, while it also discusses future AMD products. It does not provide independent testing comparing real-world performance across these designs. Memory-bandwidth specifications alone do not establish how quickly a model will run, since software, workload, power limits and the rest of the system also matter.

What to Watch in Production

The next developments to watch are HBM supply, packaging capacity and product availability. TSMC’s stated plans for larger CoWoS-L packages and suppliers’ progress on denser memory stacks will indicate how quickly the industry can produce increasingly large accelerator assemblies. The report provides no new production schedule for those expansions beyond the statements it cites.

For buyers and operators, future system specifications will show how these package designs translate into complete racks, including power demand, cooling requirements, GPU counts and delivery timing. The article’s 3D models are an explanatory view of the hardware, not a forecast of when particular systems will become broadly available.

Key Questions

What is inside an AI accelerator package?

It can include compute dies, HBM stacks, a silicon interposer and a package substrate, with a cooling plate fitted above the package in a server. Exact layouts vary by product.

What does HBM do?

High-bandwidth memory stacks DRAM chips and connects them vertically with TSVs. Placing the stacks close to compute dies provides many short connections for moving data.

Why does the interposer matter?

The silicon interposer provides dense wiring between compute dies and memory. Advanced packaging makes it possible to connect components that would be difficult to link with a conventional circuit board.

Does the article’s model show exact chip dimensions?

No. The article says the models show layer order and component counts, but use schematic sizes because manufacturers do not publish the measurements needed for an exact dimensional model.

How many GPUs are in an Nvidia NVL72 rack?

The configurations described in the report contain 72 GPUs. Nvidia’s GB200 NVL72 description also lists 36 Grace CPUs, along with compute trays, NVLink switch trays, management switches and power shelves.

Source: ThorstenMeyerAI.com

You May Also Like

Major Airline Grounds Fleet After AI Scheduling Glitch

Grounding its entire fleet due to an AI scheduling glitch, the airline’s safety measures reveal crucial lessons about automation risks.

Fervo Raises Nearly $2 Billion in IPO

Fervo has raised close to $2 billion in its initial public offering, marking a significant milestone for the geothermal energy company amid growing renewable energy investments.

Why OpenAI And Poland’s NASK Are Collaborating On AI Safety

OpenAI and Poland’s NASK announced an AI safety and security partnership, but have not specified projects, funding, timing or measures of progress.

Blender 5.3 Released With Gaussian Splat Support

Blender 5.3 can import and render Gaussian Splats, alongside updates to Cycles, EEVEE, the compositor and Geometry Nodes.