The next big breakthrough in tech products will come from chip stacking technology

The Wall Street Journal published an article saying that the next major breakthrough in technology products will occur in the field of chip stacking.

Apple Watch uses advanced 3D chip stacking packaging technology

Something interesting is happening with microchips, the building blocks of nearly every everyday electronic product. Usually thin and flat, they are now being stacked like pancakes, going from two dimensions to three dimensions — with major implications for electronic devices.

Chip designers are discovering that stacking can bring all sorts of unexpected benefits in performance, energy consumption and functionality.

Without this technology, the Apple Watch wouldn’t be possible, nor would Samsung’s state-of-the-art solid-state storage, artificial intelligence systems from Nvidia and Google, and Sony’s super-fast new cameras.

This 3D stacking is akin to city planning. Without it, microchips on a circuit board would stretch out as more parts are built into a product, with the microchips getting farther and farther apart. Once you start stacking chips, however, you can form a silicon "city" where everything is closer together.

From a physics perspective, the advantages of this design are obvious: When electrons need to travel longer distances through copper wires, they consume more energy, generate heat, and reduce bandwidth. Greg Yeric, director of future silicon technology at ARM Research, a microchip design company under ARM, pointed out that stacked chips are more efficient, generate less heat, and can communicate at the speed of light in much shorter interconnect channels.

X-ray image of the Apple S1 chip in the Apple Watch Series 1

While the principle behind 3D stacked chips is straightforward, manufacturing them is not easy. The concept of the technology was first proposed in the 1960s and has since appeared sporadically in some high-end applications, such as military hardware, Yerick said.

However, Sinjin Dixon-Warren, an analyst at microchip research firm TechInsights, points out that stacked chip products from most of the big chipmakers (AMD, Intel, Apple, Samsung and Nvidia) as well as smaller specialist companies such as Xilinx have only been around for about five years. Why do people do this? Because engineers are starting to find no other way to make chips perform better.

Stacked chips are usually part of a "package" of other chips curled up together. In addition to saving space, this allows manufacturers to build many different chips (through different manufacturing processes) and then more or less glue them together. The "3D stacked package" approach is different from the "system-on-a-chip" approach frequently used in mobile phones, which etches all the different phone components onto a single piece of silicon.

Dickson-Warren said that since the first generation, Apple Watch has been driven by one of the most advanced 3D stacked chip packages. In the smartwatch, 30 different chips are sealed in a plastic envelope. He said that to save space, memory chips are stacked on top of logic circuits. Without chip stacking technology, the watch's design cannot be made so compact.

While Apple's chips are stacked only two layers high, Samsung has built a veritable silicon skyscraper. Samsung's V-NAND flash memory, which is used to store data in phones, cameras and laptops, has 64 layers of chips. Samsung just announced that future versions will have 96 layers.

Nvidia's Volta microprocessor for artificial intelligence has eight layers of high-bandwidth memory stacked on the GPU

Memory is a natural application for chip stacking because it solves a problem that has long plagued chip designers: adding more cores to anything from an iPad to a supercomputer doesn't yield the desired speed boost because of the communication latency between the logic circuits and the required memory capacity. Stacking memory components directly on the chip shortens the path between the two.

That’s how Nvidia’s Volta microprocessors, built for AI, work, said Brian Kelleher, Nvidia’s senior vice president of hardware engineering. By stacking eight layers of high-bandwidth memory directly on top of the GPU, the chips set new records in processing efficiency.

“We are power-constrained,” Kehler said. “Any power we can free up from storage we can use for compute.”

Chip stacking also opens up new capabilities. Some phone cameras stack image sensors directly on top of the chips that process the images. The extra speed means they can take multiple exposures of a photo and blend them together to capture more light in dimly lit scenes.

Samsung's 64-layer V-NAND vertical chip has greater data storage capacity and faster processing speed

The prototype camera from Sony goes a step further by using three layers of chips instead of two - including image sensor, memory and logic circuits, to achieve up to 1,000 frames per second. What this does is that light hits the image sensor and the data goes directly to the memory, where it is then processed in real time. In addition to achieving greater visibility in low-light conditions, this can also be used to shoot super slow-motion video, freezing fast-moving objects in a single frame.

Currently, it takes enormous resources to overcome some obstacles to bring 3D microchips to more electronic devices.

First, 3D chips are new, and the design tools for stacking them haven’t evolved enough, Yerick said. Until simple design tools — similar to those used to flatten chips — become widely available, stacked chips will remain a possibility only for companies with top engineering talent.

Another problem is that manufacturers are still learning how to physically stack and connect chips to each other reliably, which means some manufacturing processes have relatively low yields.

However, Dixon-Warren pointed out that the popularity of 3D stacked chips is very fast, and they will inevitably become the mainstream of the industry. Ten years ago, the technology existed almost only in university laboratories; five or six years ago, it was difficult to find its commercialization cases. But now it has sprung up like mushrooms after rain, appearing in various applications such as networking, high-performance computing and high-end wearable devices such as Apple Watch. According to Kyle Wiens, CEO of iFixit, a well-known electronic product disassembly website, it also appears in the "brain" of iPhone X.

Eventually, 3D chips should make wearables as powerful as larger devices, allowing them to run for days on end, even if they're covered in sensors, according to ARM's Jerrick. "I wouldn't be surprised if your watch could one day check your blood sugar levels, for example," he said.

Making chips three-dimensional from two dimensions is just the beginning. Soon, chip layers will communicate via light rather than electricity. Even further in the future, they’ll move away from silicon altogether—perhaps toward synthetic diamonds—as we replace circuit boards with shiny crystals that have unprecedented processing power.