According to Business Korea, the memory modules were stuffed into 96 AMD Instinct MI100 GPUs and increased AI training performance by 2.5 times.

For those who came in late, PIM is a new generation of computer memory that can speed up computationally complex workflows handled by processors such as CPUs and GPUs. Each memory module processes data independently which slashes the amount of data needed to travel between the memory and the processor.

Samsung demonstrated the PIM-modified GPUs in October, but the combined 96 PIM-modified GPUs cluster is new. Compared to normal video memory, these modified MI100 chips not only performed 2.5 times better, they also cut power consumption by 2.67 times.

When Samsung trained the language model algorithm T5 with this system, its performance improved by 2.5 times and its power consumption dropped by 2.67 times, compared to when PIMs were not used.

The company has been developing PIM for some time now. The company demoed several implementations in 2021, involving several different memory types including DDR4, LPDDR5X, GDDR6, and HBM2. In LPDDR5 form,

Samsung saw a 1.8 times increase in performance with a 42.6 per cent reduction in power consumption and a 70 per cent reduction in latency on a test program involving a Meta AI workload. This was based on a standard server system with no modifications to the motherboard or CPU.

PIM is starting to look like it has legs when it comes to speeding up AI-accelerated workflows.