This is a solution for Tesla supercomputers
Tesla has its own supercomputer, Dojo, which is unique in that it is based on Tesla's own chips. But the company is already working on the new generation of Dojo processors, and this is a solution of gigantic proportions.
Actually a new Dojo chip — it's almost a whole 300mm plate cut to a square shape. About the same as in the case of Cerebras chips.
At the same time, the structure of the Tesla chip is clearly visible — It’s still 25 separate identical chips that simply weren’t separated. This approach allows you to increase the speed of internal interchip connections and reduce latency, although there is a disadvantage in the form of using only the memory that is implemented directly on this giant chip.
Training tiles are known to require a lot of energy. To power the chip, Tesla uses a very complex voltage regulation module that supplies 18,000 A of power. And it has to dissipate up to 15 kW of power.