UK Based AI Computer Company, Graphcore, Unveils Next-Generation AI Chip Called ‘Bow’ To Speed AI By 40% 

Graphcore, a U.K.-based AI computer company, improved the performance of its computers without changing anything about their specialized AI processor cores. TSMC’s wafer-on-wafer 3D integration technology was used to attach a power-delivery chip to Graphcore’s AI processor during manufacturing. 

According to Graphcore, its new combination chip, codenamed Bow, is the first on the market to employ wafer-on-wafer bonding. The Bow can now run at a higher frequency—1.85 GHz versus 1.35 GHz—and at a lower voltage than its predecessor because of power-delivery silicon. This results in machines that can train neural networks up to 40% quicker while using up to 16% less energy than prior generations. Users benefit from this enhancement without making any changes to their software. 

Multiple silicon die will be joined together to augment the performance gains achieved from rising progress along an ever-slowing Moore’s Law path. Bow and the Colossus MK2 (its predecessor) were produced with TSMC’s N7 manufacturing technology. 

Extracted chips are linked to other chips or wafers in various 3D chip stacking technologies, such as Intel’s Foveros. Two whole wafers of chips are bonded in TSMC’s SoIC WoW technology. When the wafers are aligned, the copper pads on the chips meet up. Pads ignite when the two wafers are pushed together. One may think of it as a cold weld between the pads. The bonded wafer is then sliced up into chips after flattening the top wafer to just a few micrometers.

One wafer has 1,472 IPU cores and 900 megabytes of on-chip memory from Graphcore’s second-generation AI processor (the firm calls them IPUs for intelligence processing units). These processors were already in commercial use and performed well in the most recent set of MLPerf testing. A comparable set of power-delivery devices was found on the opposite wafer. There are no transistors or other active components on these devices. Instead, they’re jam-packed with capacitors and through-silicon vias, which are vertical connections. The latter provides power and data connections to the processor die through the power chip.

The capacitors are the ones that truly make a difference. Like the bit-storing capacitors in DRAM, these components are created in deep, narrow trenches in silicon. The power supply is smoothed out by positioning these charge reservoirs near the transistors, allowing the IPU cores to function quicker at a lower voltage. The IPU would have to elevate its operational voltage over its nominal level to run at 1.85 GHz without the power-delivery device, using more power. The power chip can attain that clock rate and consume less energy.

Wafer-on-wafer technology allows for a higher density of connections between processors than mounting individual chips to a wafer. However, the “known good die” problem was a long-standing difficulty with this approach. There are always a few defective chips in a batch of wafers. By fusing two wafers together, the number of faulty chips might be doubled. 

Graphcore’s solution is to let it happen to some extent. The IPU, like several other emerging AI processors, is made up of a lot of identical (and hence redundant) CPU cores and other elements. Any duds may be switched off from the rest of the IPU using built-in fuses.

Although the new product’s power-delivery device lacks transistors, they may be on the way. Using the technology alone for power distribution is “simply the first step.” In the not-too-distant future, it will go considerably further.

Supercomputers capable of training “brain-scale” AIs, which are neural networks with hundreds of trillions of parameters, may be developed using this technology soon. The “Good” computer, named for British scientist IJ “Jack” Good, would have a processing power of more than 10 exaflops, or ten billion billion floating-point operations. 512 systems with 8,192 IPUs and mass storage, CPUs, and networking would make up Good. It will have a memory capacity of 4 petabytes and a bandwidth of more than 10 petabytes per second. Each supercomputer is expected to cost around $120 million and be ready for delivery in 2024, according to Graphcore.

References:

• https://3dfabric.tsmc.com/english/dedicatedFoundry/technology/SoIC.htm#SoIC_WoW
• https://spectrum.ieee.org/graphcore-ai-processor
• https://www.graphcore.ai/
• https://venturebeat.com/2022/03/03/graphcore-unveils-next-generation-ai-chips-and-plans-for-powerful-supercomputer/
• https://www.forbes.com/sites/karlfreund/2022/03/03/graphcore-launches-3rd-gen-ai-with-wafer-on-wafer-wow-technology/?sh=56703644350f
• https://www.zdnet.com/article/ai-computer-maker-graphcore-unveils-3-d-chip-promises-500-trillion-parameter-ultra-intelligence-machine/

Suggested