The number of cores increased to 900,000
Cerebras has unveiled the third generation of its monster Wafer Scale Engine (WSE). Its dimensions have not become larger, since there is simply nowhere to grow, but its characteristics have improved greatly.
Recall that the entire 300mm semiconductor wafer is used to create a chip, resulting in a chip with an area of more than 46,000 mm2.
WSE-3 uses the same principle, since the market has not yet switched to 400 mm wafers. As a result, the chip area is the same as in the first two versions, but thanks to the transition to the 5 nm process technology (from 7 nm), the parameters have become even more impressive. True, relative to the second generation, the growth is not as huge as during the transition from the first to the second.
So, if WSE-1 had 400,000 cores, and WSE-2 — already 800,000 cores, then WSE-3 offers 900,000 cores. At the same time, the number of transistors has grown much more significantly: from 2.6 trillion to 4 trillion.
Memory has increased from 40 to 44 GB, and bandwidth has increased from 20 to 21 PB/s. All this provides AI performance of 125 PFLOPS and is capable of training models with 24 trillion parameters.
As in the previous generation, the company immediately offers a supercomputer. It is called CS-3, and is a system with one single WSE-3 chip, but supplemented by a cooling system, storage drives and power supplies. Such a setup, according to the company, is capable of training models that are 10 times larger than GPT-4 and Gemini.
It is already known that 64 CS-3 installations will be used for the Condor Galaxy 3 supercomputer, which will provide 8 ExaFLOPS artificial intelligence computing performance.