The Technology section is published with the support of Favbet Tech
Computer Science researchers have discovered a new way to multiply large matrices faster, eliminating previously unknown inefficiencies, resulting in the biggest improvement in matrix multiplication efficiency in a decade. This could eventually speed up artificial intelligence models that rely heavily on matrix multiplication.
Multiplying two rectangular numeric arrays, known as matrix multiplication, plays a critical role in modern AI models, including speech and image recognition, and is used by chatbots from all major developers, AI image generators, and synthesis models video. Beyond AI, matrix mathematics is essential to modern computing (think image processing and data compression), where even small improvements in efficiency can lead to computational and energy savings, reports ArsTechnica.
Graphics processing units (GPUs) perform matrix multiplication tasks due to their ability to process many calculations simultaneously. They break large matrix problems into smaller segments and solve them simultaneously using an algorithm.
Improving this algorithm was key to breakthroughs in the efficiency of matrix multiplication over the last century—even before computers existed. In October 2022, a new technique was invented by Google DeepMind's AI model called AlphaTensor, focusing on practical algorithmic improvements for specific matrix sizes, such as 4×4 matrices.
In contrast, new research from a team of researchers from Tsinghua University, UC Berkeley, and MIT aims to make theoretical improvements to reduce the complexity index ω to improve efficiency for all matrix sizes. Rather than looking for immediate practical solutions such as AlphaTensor, the new technique aims for fundamental improvements that could change the efficiency of matrix multiplication on a more general scale.
The traditional method of multiplying two n by n matrices requires n³ separate multiplications. However, a new technique that improves on the “laser method” introduced by Volker Strassen in 1986 has reduced the upper limit of the exponent (denoted as the aforementioned ω), bringing it closer to the ideal value of 2, which represents the theoretical minimum number of operations required.
Power Skills For Tech. Reduce the amount of confusion at work and become more effective in a multicultural team. Take a 10% discount with promo code ITCENG. Registration for the course
The traditional way of multiplying two grids filled with numbers can require performing calculations up to 27 times for a 3×3 grid. But with these improvements, the process is speeded up by greatly reducing the required multiplication steps. This minimizes the number of operations to just over twice the size of one side of the squared mesh, adjusted by a factor of 2.371552. This is very important because it almost achieves the optimal efficiency of doubling the size of the square, which is the fastest speed we could ever hope for.
The 2023 breakthrough was made possible by the discovery of “hidden waste” in the laser method, where useful blocks of data were inadvertently discarded. In the context of matrix multiplication, “blocks” refer to smaller segments into which a larger matrix is divided for ease of processing, and “block labeling” is a technique for categorizing these segments to determine which ones to keep and which ones to discard, optimizing the multiplication process for speed and efficiency. By modifying the way the blocks were labeled, the researchers were able to reduce waste and significantly increase efficiency.
What practical application does this have? For AI models, reducing the number of computational steps in matrix mathematics can lead to shorter training times and more efficient task execution. This will allow more complex models to be trained faster, potentially leading to improved AI capabilities and the development of more complex AI applications. In addition, increased efficiency can make AI technologies more accessible by reducing the processing power and energy consumption required to perform these tasks.
The exact impact on the speed of AI models depends on the specific architecture of the system and how heavily its tasks rely on matrix multiplication. Increasing algorithmic efficiency often needs to be combined with hardware optimization to fully realize potential speed gains. But still, over time, as improvements in algorithmic methods accumulate, AI will become faster.
The Technology section is published with the support of Favbet Tech
Favbet Tech is IT a company with 100% Ukrainian DNA, which creates perfect services for iGaming and Betting using advanced technologies and provides access to them. Favbet Tech develops innovative software through a complex multi-component platform that can withstand enormous loads and create a unique experience for players. The IT company is part of the FAVBET group of companies.
The competition for ITS authors continues. Write an article about the development of games, gaming and gaming devices and win a professional gaming wheel Logitech G923 Racing Wheel, or one of the low-profile gaming keyboards Logitech G815 LIGHTSYNC RGB Mechanical Gaming Keyboard!