Californian start-up Cerebras Systems has shown off what it claims to be the "world's largest computer chip" at the Hot Chips conference at Stanford University in Palo Alto, California. This AI processor, dubbed the Wafer Scale Engine (WSE), contains 1.2 trillion transistors. While most chips are small portions of a 12-inch silicon wafer this is a single chip interconnected on a single wafer.
A team headed by Andrew Feldman, previously of SeaMicro which was sold to AMD in 2012. Feldman works alongside co-founder Sean Lie and they provided an overview of their chip at Hot Chips. Large chips with masses of cores are said to be particularly adept at AI processing. The WSE boasts 400,000 cores over its 21.5cm sq (8.5inch sq). It is thus 56x bigger than the largest Nvidia GPU. Furthermore, the WSE was 3,000 times more on-chip memory and 10,000 times the bandwidth of Nvidia's biggest GPU.
Other interesting stats are that the WSE packs no cache, 18GB of local superfast SRAM, and has a memory bandwidth of 9 petabytes per second. The cores are linked with a fine-grained, all-hardware, on-chip mesh-connected communication network that delivers an aggregate bandwidth of 100 petabits per second.
"Designed from the ground up for AI work, the Cerebras WSE contains fundamental innovations that advance the state-of-the-art by solving decades-old technical challenges that limited chip size — such as cross-reticle connectivity, yield, power delivery, and packaging," said Feldman. He added that the architecture is optimised for AI, and it can deliver, depending upon the workload, "hundreds or thousands of times the performance of existing solutions at a tiny fraction of the power draw and space."
There are numerous technical challenges to making wafer scale processors. Cerebras had to implement high-speed die-to-die communication, it worked around manufacturing defects, on the packaging of such a large chip, and providing high-density power and cooling for it.
While this might be the largest computer chip, Samsung has previously built a flash memory chip with 2 trillion transistors. The Cerebras WSE computer chip was fabricated by TSMC.
